Skip to content

Commit 2c23d6f

Browse files
Andy-Jostclaude
andcommitted
cuda.core: address the self-review of host_launch
State that a ctypes callback that wraps a Python function acquires the GIL through ctypes, so the deadlock rule applies to it; complete the Raises section; say the driver may reject CUDA calls from the callback rather than that it does. Upcast the launch record to the cleanup base before the void* parameter, as the graph producers do. Prove the user_data copy by changing the caller's buffer while the stream is held, pin the unraisable type and message, and link the example from the examples page. Co-Authored-By: Claude Fable 5.1 <[email protected]>
1 parent 426ec5a commit 2c23d6f

8 files changed

Lines changed: 51 additions & 25 deletions

File tree

‎cuda_core/cuda/core/_cpp/rt/DESIGN.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ cuda/core/
124124
├── context_scope.hpp # Scoped-context helpers (namespace detail)
125125
├── internal.hpp # Registry, cleanup wrappers, deferred-cleanup item (namespace detail)
126126
├── py_report.cpp, py_deferred_cleanup.cpp # Python-coupled bodies
127-
├── host_launch.cpp # The stream-side host function launch record
127+
├── host_launch.cpp # The stream-side host function launch record
128128
└── context.cpp, stream.cpp, event.cpp, memory.cpp, program.cpp,
129129
graph.cpp, graph_exec.cpp, texture.cpp # One resource family per file
130130
```

‎cuda_core/cuda/core/_cpp/rt/GRAPH_ATTACHMENTS.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,8 @@ deleters call CUDA or run Python finalizers.
127127
destructor callback only adds the attachment to the process-lifetime
128128
`DeferredCleanupQueue` and requests a `Py_AddPendingCall`.
129129

130-
The host launch record of `host_launch.cpp` is the queue's second producer;
130+
The host launch record of `host_launch.cpp` is the one producer that is not a
131+
graph attachment;
131132
its trampoline enqueues the record after the user function returns.
132133

133134
One pending call drains all queued attachments from Python's main thread. The

‎cuda_core/cuda/core/_cpp/rt/host_launch.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ host_launch_trampoline(void *p) noexcept
3131
auto *rec = static_cast<HostLaunchRecord *>(p);
3232
rec->fn(rec->user_data);
3333
// Owners may run Python finalizers or CUDA calls, so release them on the main thread.
34-
detail::enqueue_cleanup(rec);
34+
detail::enqueue_cleanup(static_cast<detail::DeferredCleanupItem *>(rec));
3535
}
3636

3737
} // namespace

‎cuda_core/cuda/core/_launcher.pyi‎

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -47,9 +47,12 @@ def host_launch(stream: Stream | GraphBuilder | IsStreamType, fn, *, user_data=N
4747
fn : callable or ctypes function pointer
4848
A Python callable taking no arguments, which runs with the GIL, or a
4949
ctypes function pointer declared as ``CUhostFn``, for example
50-
``ctypes.CFUNCTYPE(None, ctypes.c_void_p)``, which the driver calls
51-
directly without the GIL. ``cuda.core`` keeps ``fn`` alive until the
52-
function has run and releases it afterwards on the main thread.
50+
``ctypes.CFUNCTYPE(None, ctypes.c_void_p)``, which ``cuda.core``
51+
passes to the driver as is. A pointer to compiled code runs without
52+
the GIL; a ctypes callback that wraps a Python function acquires the
53+
GIL through ctypes and is subject to the same deadlock rule as a
54+
Python callable. ``cuda.core`` keeps ``fn`` alive until the function
55+
has run and releases it afterwards on the main thread.
5356
user_data : int or bytes-like, optional
5457
Only for ctypes function pointers. An ``int`` is a raw pointer that
5558
the caller must keep valid until the function has run. Bytes-like data
@@ -59,11 +62,13 @@ def host_launch(stream: Stream | GraphBuilder | IsStreamType, fn, *, user_data=N
5962
Raises
6063
------
6164
TypeError
62-
If ``fn`` is not callable, or its ctypes prototype is not ``CUhostFn``.
65+
If ``stream`` is ``None`` or not a stream, or if ``fn`` is not
66+
callable or its ctypes prototype is not ``CUhostFn``.
6367
ValueError
6468
If ``user_data`` is given for a Python callable.
6569
RuntimeError
66-
If a :class:`Stream` is capturing a graph.
70+
If ``stream`` is a :class:`Stream` that is capturing a graph, or a
71+
:class:`~graph.GraphBuilder` that is closed or not building.
6772
6873
Notes
6974
-----
@@ -73,9 +78,10 @@ def host_launch(stream: Stream | GraphBuilder | IsStreamType, fn, *, user_data=N
7378
.. warning::
7479
7580
The function must not call CUDA, directly or through a library. The
76-
driver rejects such calls with an error.
81+
driver does not permit such calls and may reject them with an error.
7782
78-
A Python callable acquires the GIL on the driver's thread. A thread
83+
A Python callable, or a ctypes callback that wraps a Python function,
84+
acquires the GIL on the driver's thread. A thread
7985
that holds the GIL while blocking inside a CUDA call waiting for this
8086
stream therefore deadlocks with the callback. ``cuda.core`` and
8187
``cuda.bindings`` release the GIL around CUDA calls. A library that

‎cuda_core/cuda/core/_launcher.pyx‎

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -90,9 +90,12 @@ def host_launch(stream: Stream | GraphBuilder | IsStreamType, fn, *, user_data=N
9090
fn : callable or ctypes function pointer
9191
A Python callable taking no arguments, which runs with the GIL, or a
9292
ctypes function pointer declared as ``CUhostFn``, for example
93-
``ctypes.CFUNCTYPE(None, ctypes.c_void_p)``, which the driver calls
94-
directly without the GIL. ``cuda.core`` keeps ``fn`` alive until the
95-
function has run and releases it afterwards on the main thread.
93+
``ctypes.CFUNCTYPE(None, ctypes.c_void_p)``, which ``cuda.core``
94+
passes to the driver as is. A pointer to compiled code runs without
95+
the GIL; a ctypes callback that wraps a Python function acquires the
96+
GIL through ctypes and is subject to the same deadlock rule as a
97+
Python callable. ``cuda.core`` keeps ``fn`` alive until the function
98+
has run and releases it afterwards on the main thread.
9699
user_data : int or bytes-like, optional
97100
Only for ctypes function pointers. An ``int`` is a raw pointer that
98101
the caller must keep valid until the function has run. Bytes-like data
@@ -102,11 +105,13 @@ def host_launch(stream: Stream | GraphBuilder | IsStreamType, fn, *, user_data=N
102105
Raises
103106
------
104107
TypeError
105-
If ``fn`` is not callable, or its ctypes prototype is not ``CUhostFn``.
108+
If ``stream`` is ``None`` or not a stream, or if ``fn`` is not
109+
callable or its ctypes prototype is not ``CUhostFn``.
106110
ValueError
107111
If ``user_data`` is given for a Python callable.
108112
RuntimeError
109-
If a :class:`Stream` is capturing a graph.
113+
If ``stream`` is a :class:`Stream` that is capturing a graph, or a
114+
:class:`~graph.GraphBuilder` that is closed or not building.
110115

111116
Notes
112117
-----
@@ -116,9 +121,10 @@ def host_launch(stream: Stream | GraphBuilder | IsStreamType, fn, *, user_data=N
116121
.. warning::
117122

118123
The function must not call CUDA, directly or through a library. The
119-
driver rejects such calls with an error.
124+
driver does not permit such calls and may reject them with an error.
120125

121-
A Python callable acquires the GIL on the driver's thread. A thread
126+
A Python callable, or a ctypes callback that wraps a Python function,
127+
acquires the GIL on the driver's thread. A thread
122128
that holds the GIL while blocking inside a CUDA call waiting for this
123129
stream therefore deadlocks with the callback. ``cuda.core`` and
124130
``cuda.bindings`` release the GIL around CUDA calls. A library that

‎cuda_core/docs/source/examples.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@ Kernels and launch configuration
3535
partitions SMs across kernels with green contexts.
3636
- :sample:`kernelNsysProfile <kernelNsysProfile/>`
3737
annotates kernel launches with NVTX markers for Nsight Systems.
38+
- :cuda-core-example:`host_launch.py <host_launch.py>`
39+
orders a host function between two kernel launches on a stream and reads a
40+
pinned buffer from it.
3841

3942
Memory management and interop
4043
-----------------------------

‎cuda_core/docs/source/release/1.3.0-notes.rst‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -47,9 +47,10 @@ New features
4747

4848
- Added :func:`host_launch`, which enqueues a host function on a stream in stream
4949
order: a Python callable, or a ctypes function pointer with the ``CUhostFn``
50-
prototype that the driver calls without the GIL. It is the stream-side
51-
counterpart of :meth:`graph.GraphBuilder.callback` and accepts the same
52-
``fn`` and ``user_data`` forms.
50+
prototype, which ``cuda.core`` passes to the driver as is, so a pointer to
51+
compiled code runs without the GIL. It is the stream-side counterpart of
52+
:meth:`graph.GraphBuilder.callback` and accepts the same ``fn`` and
53+
``user_data`` forms.
5354
(`#2058 <https://github.com/NVIDIA/cuda-python/issues/2058>`__)
5455

5556
- Added incremental linking to :class:`Linker` through

‎cuda_core/tests/test_host_launch.py‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44

55
import ctypes
66
import sys
7+
import threading
78
import time
89
import weakref
910
from contextlib import closing
@@ -69,18 +70,26 @@ def __call__(self):
6970

7071
@pytest.mark.agent_authored(model="gpt-6.1-sol")
7172
def test_host_launch_ctypes_bytes_user_data_is_copied(init_cuda):
73+
"""The bytes are copied at launch; a later change to the caller's buffer is not seen."""
7274
dev = Device(0)
7375
dev.set_current()
7476
seen = []
77+
gate = threading.Event()
7578

7679
@ctypes.CFUNCTYPE(None, ctypes.c_void_p)
7780
def callback(data):
7881
seen.append(ctypes.string_at(data, N))
7982

8083
with closing(dev.create_stream()) as stream:
81-
data = bytes(range(N))
82-
host_launch(stream, callback, user_data=data)
83-
del data
84+
data = bytearray(range(N))
85+
try:
86+
# Hold the stream on the driver thread (Event.wait releases the GIL) while
87+
# the caller overwrites its buffer; the copy made at launch is unaffected.
88+
host_launch(stream, gate.wait)
89+
host_launch(stream, callback, user_data=data)
90+
data[:] = b"\xff" * N
91+
finally:
92+
gate.set()
8493
stream.sync()
8594
assert seen == [bytes(range(N))]
8695

@@ -181,8 +190,8 @@ def throws():
181190
host_launch(stream, lambda: ran.append(True))
182191
stream.sync()
183192
assert ran == [True]
184-
assert len(records) == 1
185-
assert records[0].exc_type is RuntimeError
193+
assert [record.exc_type for record in records] == [RuntimeError]
194+
assert "callback raised" in str(records[0].exc_value)
186195

187196

188197
@pytest.mark.agent_authored(model="gpt-6.1-sol")

0 commit comments

Comments
 (0)