Skip to content

Commit 6cb1826

Browse files
Andy-Jostclaude
andcommitted
docs(cuda.core): restore __weakref__ on VirtualMemoryResource and state the release wait
The Python class had __weakref__ as a subclass of an extension type; the cdef class declares it, as Buffer and the pool-backed resources do. VMM_DESIGN.md compared the blocking release to pool-backed deleters, which do not block: cuMemFreeAsync is stream-ordered. The precedent is _SynchronousMemoryResource and LegacyPinnedMemoryResource, which wait in deallocate() on the same deleter path. The design doc, the class docstring, close() and the release note now say that closing a buffer waits for the work on its deallocation stream and how to control when that happens. The release note also drops a sentence about the range deleter synchronizing every recorded stream, which immutable ranges made false. The follow-up for a stream-ordered release is NVIDIA#2989. Co-Authored-By: Claude Fable 5.1 <[email protected]>
1 parent 53479be commit 6cb1826

4 files changed

Lines changed: 36 additions & 13 deletions

File tree

‎cuda_core/cuda/core/_cpp/rt/VMM_DESIGN.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -124,9 +124,15 @@ the call, so a capture on an unrelated stream is not invalidated. Then drop the
124124
mapping this buffer was the last to hold unmaps, and its reservation frees and its allocation
125125
releases as their last references go. This is the same model as every other `Buffer`: the
126126
recorded stream orders the release of this buffer's memory, and a caller who touches that memory
127-
from another stream must order that work before the close. The deleter blocks on the sync, like
128-
the pool-backed deleters block on the free, and it may run inside the deferred-cleanup drain on
129-
the main thread, with the GIL released.
127+
from another stream must order that work before the close. The deleter blocks until that work
128+
completes. `cuMemUnmap` is not stream-ordered, so waiting is the only way to honor the order;
129+
`_SynchronousMemoryResource` and `LegacyPinnedMemoryResource` wait the same way in
130+
`deallocate()`, while pool-backed buffers never block because `cuMemFreeAsync` is stream-ordered.
131+
The wait happens wherever the last reference goes: an explicit `close()`, a garbage collection,
132+
or the deferred-cleanup drain on the main thread, with the GIL released. A caller who needs to
133+
control when it happens closes the buffer explicitly or records an idle deallocation stream.
134+
Emulating stream order with a host callback that hands the release to the deferred-cleanup
135+
queue is planned as a follow-up for every synchronous release in cuda.core.
130136

131137
Allocations are shared by two ranges after a grow that moves the buffer. Shared ownership is
132138
what makes that safe: the allocation is released exactly once, when its last mapping goes.

‎cuda_core/cuda/core/_memory/_virtual_memory_resource.pyi‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -94,10 +94,11 @@ class VirtualMemoryBuffer(Buffer):
9494
"""Release this buffer's share of its address range.
9595
9696
Each mapping, reservation and physical allocation goes away when the
97-
last buffer that holds it closes. Before this buffer's share is
98-
released, its deallocation stream is synchronized, as for every
99-
:class:`Buffer`. Virtual memory deallocation is synchronous and cannot
100-
be captured. When the stream this close uses, given or recorded, is not
97+
last buffer that holds it closes. Virtual memory is unmapped
98+
synchronously, so this call waits for the work queued on the
99+
deallocation stream before it releases this buffer's share; a buffer
100+
released by the garbage collector waits at that point instead. The
101+
unmap cannot be captured. When the stream this close uses, given or recorded, is not
101102
a default stream and is capturing, the call raises and leaves the
102103
buffer open. A default stream is checked when the range is released
103104
instead: if synchronizing it would disturb a capture in its context,
@@ -134,6 +135,12 @@ class VirtualMemoryResource(MemoryResource):
134135
Every buffer this resource returns is a :class:`VirtualMemoryBuffer` that
135136
owns its address reservations, physical allocations and mappings; closing
136137
the buffer releases them. :meth:`deallocate` is not involved in that path.
138+
139+
Virtual memory is unmapped synchronously, so closing a buffer waits for
140+
the work queued on its deallocation stream. A buffer released by the
141+
garbage collector waits at that point instead. To control when the wait
142+
happens, close the buffer explicitly or record an idle stream with
143+
:meth:`Buffer.set_deallocation_stream`.
137144
"""
138145
device: object
139146
config: object

‎cuda_core/cuda/core/_memory/_virtual_memory_resource.pyx‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -229,10 +229,11 @@ cdef class VirtualMemoryBuffer(Buffer):
229229
"""Release this buffer's share of its address range.
230230

231231
Each mapping, reservation and physical allocation goes away when the
232-
last buffer that holds it closes. Before this buffer's share is
233-
released, its deallocation stream is synchronized, as for every
234-
:class:`Buffer`. Virtual memory deallocation is synchronous and cannot
235-
be captured. When the stream this close uses, given or recorded, is not
232+
last buffer that holds it closes. Virtual memory is unmapped
233+
synchronously, so this call waits for the work queued on the
234+
deallocation stream before it releases this buffer's share; a buffer
235+
released by the garbage collector waits at that point instead. The
236+
unmap cannot be captured. When the stream this close uses, given or recorded, is not
236237
a default stream and is capturing, the call raises and leaves the
237238
buffer open. A default stream is checked when the range is released
238239
instead: if synchronizing it would disturb a capture in its context,
@@ -288,11 +289,18 @@ cdef class VirtualMemoryResource(MemoryResource):
288289
Every buffer this resource returns is a :class:`VirtualMemoryBuffer` that
289290
owns its address reservations, physical allocations and mappings; closing
290291
the buffer releases them. :meth:`deallocate` is not involved in that path.
292+
293+
Virtual memory is unmapped synchronously, so closing a buffer waits for
294+
the work queued on its deallocation stream. A buffer released by the
295+
garbage collector waits at that point instead. To control when the wait
296+
happens, close the buffer explicitly or record an idle stream with
297+
:meth:`Buffer.set_deallocation_stream`.
291298
"""
292299

293300
cdef:
294301
readonly object device
295302
readonly object config
303+
object __weakref__
296304

297305
def __init__(self, device_id: Device | int, config: VirtualMemoryResourceOptions | None = None) -> None:
298306
self.device = Device(device_id)

‎cuda_core/docs/source/release/1.3.0-notes.rst‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -210,8 +210,10 @@ Fixes and enhancements
210210
Buffers from ``allocate()`` free themselves when they close and no longer
211211
call :meth:`VirtualMemoryResource.deallocate`, which now serves pointers
212212
wrapped with :meth:`Buffer.from_handle`. A buffer records the stream passed
213-
to ``allocate()`` and the last close of an aliased range synchronizes every
214-
recorded stream before it unmaps. ``modify_allocation`` accepts only buffers
213+
to ``allocate()``, and closing it waits for the work queued on that stream
214+
before it unmaps, because ``cuMemUnmap`` is not stream-ordered; close the
215+
buffer explicitly, or record an idle deallocation stream, to control when
216+
that wait happens. ``modify_allocation`` accepts only buffers
215217
this resource returned. ``location_type="host"`` requires
216218
``handle_type=None``, which the driver requires. ``allocate(0)`` returns an
217219
empty buffer without a driver call.

0 commit comments

Comments
 (0)