Repository navigation
[RFC] Make cuda.core a required dependency #128
Description
Recent works in cuda.core/cuda.bindings have reached to a point where we can replace most, if not all, numba-cuda internal wrappers to driver, nvvm, linker, … by cuda.core. It has been the plan to make cuda.core a required (not just optional) dependency of numba-cuda (and cuda.bindings would become a transitive dependency). There are multiple benefits, such as
- separation of concerns:
numba-cudafocuses on the compiler stack and leave CUDA’s Python exposure to CUDA Python - support CUDA wheels out of box (so issue 65 and issue 66 do not need any effort on the numba-cuda side)
- better performance (since cuda-bindings is based on Cython instead of ctypes)
- support Windows (which pynvjitlink does not, as of today)
- coverage of latest CUDA features
- …
This RFC is created to track necessary efforts:
- [FEA] Make
cuda.core.Bufferrecognized by numba-cuda #152 - [FEA] Make
cuda.core.utils.StridedMemoryViewrecognized by numba-cuda #153 - [FEA]: Make
pathfinderusable as a full replacement forcuda_paths.pyinnumba-cudacuda-python#1036 - [MNT] Replace all internal
Linker-related classes bycuda.core.(experimental).Linker#129 - [RFC] Archive this project in favor of
cuda.core.(experimental).Linkerrapidsai/pynvjitlink#130 - Support NVVM IRs as input to
Programcuda-python#452 - [FEA] Make
cuda.core.Streamrecognized by numba-cuda by supporting the__cuda_stream__protocol #151 - [MNT] Drop
CUDA_USE_NVIDIA_BINDINGand always usecuda.core(and fallback tocuda.bindings) #154 - [RFE] Better support for CUDA Minor Version Compatibility #262
- Deprecate/remove
cuda_paths.pyentirely #302
Activity
realarnavgoel commentedon Mar 6, 2025
@leofang Does
cuda.core._launchersupport option inLaunchConfigto selectcuLaunchCooperativeKernelorcuLaunchKerneldepending on the user needs forgrid.sync()or other deadlock-free guarantees when performing non-local operations in each GPU (e.g. barrier between GPUs) of a multi-GPU application ?numba-cudasupports this automatically and selects the appropriate mechanism in the dispatcher: https://github.com/NVIDIA/numba-cuda/blob/main/numba_cuda/numba/cuda/dispatcher.py#L409leofang commentedon Mar 6, 2025
The beauty of the new
cuLaunchKernelExAPI is it is not our (cuda.core, and maybe even Numba) job to choose which API to use. We just need to add a new option toLaunchConfigto hook withCU_LAUNCH_ATTRIBUTE_COOPERATIVEas you suggested, so that the driver knows to launch it cooperatively. Please open an issue in NVIDIA/cuda-python to track it.realarnavgoel commentedon Mar 6, 2025
Created an issue: NVIDIA/cuda-python#496
cpcloud commentedon Dec 1, 2025
For transparency:
cuda.core.Bufferrecognized by numba-cuda #152 is blocked by support for__cuda_array_interface__incuda.core(or DLPack support in numba-cuda, which is likely much more work).cuda.core.utils.StridedMemoryViewrecognized by numba-cuda #153 is blocked by feat: Introduce StridedLayout, support wrapping external allocations in Buffer, add StridedMemoryView.from_buffer cuda-python#1283.pathfinderusable as a full replacement forcuda_paths.pyinnumba-cudacuda-python#1036 isn't blocked on anything AFAICT, but maybe @rwgk knows more specifically if anything is blocking there?rwgk commentedon Dec 1, 2025
Correct. It's purely a matter of finding the time to work on it.
cpcloud commentedon Dec 4, 2025
I added #153 since it's likely we'll need to do that to address the RFC.
cudadrv/driver.py#645cpcloud commentedon Dec 15, 2025
#152 and #153 are now hard blocked until
StridedLayout(or something like it) exists andStridedMemoryView.from_buffer(or something like it) exists.gmarkall commentedon Dec 18, 2025
If this issue is hard blocked until these exist, it will probably not fit into the 0.24.0 milestone - moving to "next".
cpcloud commentedon Dec 18, 2025
I don't think this is hard blocked anymore. We made
_StridedLayoutprivate, butfrom_bufferstill exists, which should be enough to get us to supportBufferandStridedMemoryViewas inputs.