[FEA] Support memory resources from CCCL 3.2

[bdice]

Is your feature request related to a problem? Please describe.
Recently, RAPIDS/CCCL nightlies began failing due to a change in CCCL's memory resources. See NVIDIA/cccl#5313

Describe the solution you'd like
RMM should support CCCL's new memory resources, which are targeting CCCL 3.2. RAPIDS is currently using CCCL 3.1.

Current plan for adoption:

Allocation Interfaces

This list of tasks requires CCCL 3.1+, so we can ship these changes in 25.12.

1. Build RMM w/ 3.1 via polyfill and allocate updates (Support building with CCCL 3.1.0 #2017) (25.10)

• Need to verify that all of RAPIDS builds with CCCL 3.1 with these changes in RMM, and ask Spark to do testing with the same pre-release of CCCL 3.1. The goal is to unblock adoption of CCCL 3.1 for RAPIDS.
• Then hopefully CCCL+RAPIDS CI should work.

2. Update polyfill to new allocate signature (RMM internal refactoring) Use CCCL MR interface internally #2112 (25.12)
3. Refactor RAPIDS to use new allocate signature Migrate RAPIDS to CCCL MR interface (new allocation APIs) #2126 (25.12)
4. Deprecate old allocate signature Add deprecation warnings for legacy MR interface #2128 (25.12)
5. Remove deprecated legacy allocate interfaces Remove legacy memory resource interface in favor of CCCL interface #2150 (26.02)

Memory Resource Handling

This list of tasks requires CCCL 3.2+, so we will need to work on that migration in 26.02.

• Implement a bridge for pointer-based resources (26.02)
  • CCCL 3.2 resource ref compatibility #2176
  • [FEA] Implement a bridge to help migrate away from device_memory_resource* #2143
• Upgrade RAPIDS to use CCCL 3.2 (26.02)
  • Update to CCCL 3.2 build-planning#237
  • Update to CCCL v3.2.0 rapids-cmake#955
• Require CCCL 3.2 in RMM: Remove LIBCUDACXX_ENABLE_EXPERIMENTAL_MEMORY_RESOURCE #2223 (26.04)
• Cleanup CCCL < 3.2: Remove compatibility code for CCCL earlier than 3.2 #2248 (26.04)
• Adopt any_resource in device-resource global mapping Store any_resource in device-resource global mapping #2200 (26.02)
• Adopt any_resource in custom containers Store any_resource in device_buffer and device_uvector #2201 (26.02)
  • Containers storing RMM resource_refs should instead store cuda::mr::any_resource
• Remove RMM polyfill namespace (26.04)
  • Remove rmm::detail::polyfill in favor of CCCL concepts #2282
  • Remove rmm::detail::polyfill in favor of CCCL concepts, require CCCL >=3.3 #2283
• Convert adaptors into cuda::shared_resource (staging, 26.06)
  • Refactor logging_resource_adaptor to shared CCCL MR design #2246
  • Refactor pool_memory_resource to shared CCCL MR design #2258
  • Refactor fixed_size_memory_resource and binning_memory_resource to shared CCCL MR design #2264
  • Refactor tracking, statistics, and aligned resource adaptors to shared CCCL MR design #2265
  • Return owning any_resource from set_per_device_resource_ref #2271
  • Refactor arena_memory_resource to shared CCCL MR design #2272
  • Refactor callback_memory_resource to shared CCCL MR design #2274
  • Refactor prefetch_resource_adaptor to shared CCCL MR design #2275
  • Refactor thread_safe_resource_adaptor to shared CCCL MR design #2276
  • Refactor limiting_resource_adaptor to shared CCCL MR design #2277
  • Refactor failure_callback_resource_adaptor to shared CCCL MR design #2278
• Remove owning_wrapper (staging, 26.06)
  • Remove owning_wrapper, no longer necessary after CCCL shared_resource adoption #2285
  • Remove owning_wrapper #2286
• Update base memory resources (staging, 26.06)
  • [FEA] Migrate base memory resources to native CCCL interface #2287
  • Migrate base memory resources to native CCCL resource concept #2289

Remove device_memory_resource and legacy interface

• Migrate Python/Cython bindings from device_memory_resource* to resource_ref / any_resource
  • [FEA] Migrate Python/Cython bindings from device_memory_resource* to resource_ref / any_resource #2294
  • Migrate Python/Cython bindings from device_memory_resource to device_async_resource_ref #2300
• Add conversion operator from rmm::cuda_stream to cuda::stream_ref
  • Add implicit conversion from rmm::cuda_stream to cuda::stream_ref #2326
• Remove device_memory_resource inheritance from all C++ memory resources
  • [FEA] Remove device_memory_resource inheritance from all C++ memory resources #2295
  • Remove device_memory_resource inheritance from all resources and adaptors #2301
• Remove bridge infrastructure and device_memory_resource
  • [FEA] Remove bridge infrastructure and device_memory_resource #2296
  • Remove bridge infrastructure and device_memory_resource #2324
• Delete cccl_adaptors.hpp and use raw CCCL resource_ref types
  • Store any_resource members in polymorphic_allocator, thrust_allocator, and device_check_resource_adaptor #2340
  • [FEA] Delete cccl_adaptors.hpp and use raw CCCL resource_ref types #2323
  • Delete cccl_adaptors.hpp and use raw CCCL resource_ref types #2325

Post-tasks

• Delete is_resource_adaptor.hpp and its test usages (no longer meaningful after shared_resource adoption)
• Use cuda_mr or cuda_async_mr in tests rather than get_current_device_resource_ref (see comment)
• Check on equality docstrings after removing legacy code (see comment)
• Update documentation: remove references to device_memory_resource, do_allocate, virtual dispatch from Doxygen and Python docstrings
• Remove stale #include directives for deleted headers (device_memory_resource.hpp, device_memory_resource_view.hpp, cccl_adaptors.hpp)
• Switch adaptors to be property-agnostic (e.g. support host accessible pools) and expose Upstream& upstream_resource()
  • This isn't in scope for the core of this issue.
• Audit public symbols (see comment)
• Enforce alignment requirements: Leaf MRs do not enforce alignment limits after device_memory_resource removal #2342
• Migration guide for downstream consumers: Add migration guide for RMM 26.04 → 26.06 breaking changes #2345 / Add 26.04 to 26.06 migration guide #2344
• cuOpt follow-up for resource casting: https://github.com/NVIDIA/cuopt/pull/1035/changes#r3106422111
• RAFT follow-up for resource casting:
  • https://github.com/rapidsai/raft/pull/2996/changes#r3117043609
  • https://github.com/rapidsai/raft/pull/2996/changes#r3117055885
• Address CodeRabbit feedback from Merge staging into main: CCCL memory resource migration #2361

Update RAPIDS libraries

The merge train 🚂🚃🚃 is tracked in #2364.

Tree 1: Structured Data Processing

• Migrate RMM usage to CCCL MR design cudf#22008
• Migrate RMM usage to CCCL MR design NVIDIA/cuCascade#98
• Migrate RMM usage to CCCL MR design rapidsmpf#940
• Migrate RMM usage to CCCL MR design facebookincubator/velox#17178
• Migrate RMM usage to CCCL MR design NVIDIA/spark-rapids-jni#4468

Tree 2: Vector Search, ML, Graph, Optimization

• Migrate RMM usage to CCCL MR design ucxx#636
• Migrate RMM usage to CCCL MR design raft#2996
• Migrate RMM usage to CCCL MR design NVIDIA/cuopt#1035
• Migrate RMM usage to CCCL MR design cuvs#1990
• Migrate RMM usage to CCCL MR design cuml#7951
• Migrate RMM usage to CCCL MR design cugraph#5483
• Migrate RMM usage to CCCL MR design dmlc/xgboost#12141

[bdice]

CCCL 3.1.0 + RAPIDS 25.10 CI is working, and RAPIDS is nearly ready to adopt the new CCCL version in 25.12. This will complete task (1) above.