Skip to content

Commit ef9b712

Browse files
Andy-Jostclaude
andcommitted
cuda.core: validate and harden VirtualMemoryResource IPC imports
Address the review of the VirtualMemoryResource IPC path. Import: check every imported allocation's location against the resource's before mapping (the same device, or the same host location type) and raise a ValueError that names both; reject a descriptor with a size but no allocations; attach notes to driver failures that say which chunk failed and why a peer-controlled size fails at the map. The imported buffer no longer keeps the descriptor: the driver copies the shared allocation into a handle of its own and keeps no reference to the file descriptor, so the descriptors live only as long as the caller holds the descriptor object. Export: an imported buffer, and a buffer grown from one, cannot be exported again because the driver exports only allocations created with the requested handle type; say so instead of surfacing INVALID_VALUE. modify_allocation() rejects a config whose handle_type differs from the resource's, so every chunk of a buffer is exportable the same way. __reduce__ preserves subclasses. Tests: parametrize the memory_ipc suite over VirtualMemoryResource (VirtualMR), with the pool-only scenarios guarded or skipped; PatternGen works on the first `size` bytes of a larger buffer; the fd-leak harness asserts on the success path; new VMM-only tests for many-chunk ranges, a second device with and without peers, descriptor validation and rollback, re-export, empty buffers, and a DLPack consumer. Docs: file descriptor and memory lifetime, multiprocessing-only transport, trust boundary, device rule, re-export limit; VirtualMemoryIPCBufferDescriptor in api_private; the from_ipc_descriptor stub has its mr type again. Co-Authored-By: Claude Fable 5.1 <[email protected]>
1 parent 4ed2c03 commit ef9b712

19 files changed

Lines changed: 954 additions & 248 deletions

‎cuda_core/cuda/core/_cpp/rt/VMM_DESIGN.md‎

Lines changed: 32 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -213,20 +213,40 @@ travel because the OS handle does not expose them to the importer; a wrong size
213213
fails the importer's `cuMemMap`.
214214

215215
The importer is a `VirtualMemoryResource` of the receiving process with the
216-
same `handle_type`. `import_mem_allocation_handle` calls
217-
`cuMemImportFromShareableHandle` and wraps the result in the same box and
216+
same `handle_type`, for the device that owns the memory. `import_mem_allocation_handle`
217+
calls `cuMemImportFromShareableHandle` and wraps the result in the same box and
218218
deleter as `create_mem_allocation_handle`: the driver gives an imported handle
219219
one reference and frees the memory when all references are released and no
220-
mapping remains, so `cuMemRelease` is the right teardown for both. The access
221-
descriptors stored in the box are the importer's, built from its options for
222-
its device; the exporter's access does not travel. The import then reserves
223-
`sum(sizes)` with the importer's alignment, maps each allocation in order, and
224-
builds a range and a `VirtualMemoryBuffer` exactly as `allocate()` does, so a
225-
grown buffer imports as one contiguous range with the same byte layout, and the
226-
imported buffer frees itself through the same deleters. A descriptor imported
227-
in the exporting process yields an alias of the exporter's memory at a new
228-
address. A `VirtualMemoryResource` pickles as (device, options), which is what
229-
lets a `Buffer` pickle as (resource, descriptor).
220+
mapping remains, so `cuMemRelease` is the right teardown for both. After each
221+
import, `cuMemGetAllocationPropertiesFromHandle` must report the resource's
222+
location (the same device, or the same host location type); a mismatch is a
223+
`ValueError` before anything is mapped, because the access descriptors would
224+
describe the wrong location. The access descriptors stored in the box are the
225+
importer's, built from its options for its device and peers; the exporter's
226+
access does not travel. The import then reserves `sum(sizes)` with the
227+
importer's alignment, maps each allocation in order, and builds a range and a
228+
`VirtualMemoryBuffer` exactly as `allocate()` does, so a grown buffer imports
229+
as one contiguous range with the same byte layout, and the imported buffer
230+
frees itself through the same deleters. A descriptor imported in the exporting
231+
process yields an alias of the exporter's memory at a new address. A
232+
`VirtualMemoryResource` pickles as (device, options), which is what lets a
233+
`Buffer` pickle as (resource, descriptor).
234+
235+
File descriptors and memory. The driver copies each shared allocation into a
236+
handle of its own during the import and keeps no reference to the file
237+
descriptor, so the imported buffer does not keep the descriptor; it records
238+
only that it was imported (`is_mapped`). The descriptor's file descriptors live
239+
as long as the descriptor object, on the exporter (where the buffer caches it)
240+
and in every process that unpickled a copy, and each open descriptor keeps the
241+
physical memory allocated. A descriptor on a `Queue` pins the memory in the
242+
sender until the receiver unpickles it.
243+
244+
Re-export. The driver exports only allocations created with the requested
245+
handle type, and an imported allocation carries none, so an imported buffer,
246+
and any buffer grown from one, cannot be exported again; `ipc_descriptor` says
247+
so instead of surfacing the driver's INVALID_VALUE. A `modify_allocation`
248+
config must keep the resource's `handle_type` for the same reason: every chunk
249+
of a buffer must be exportable the same way.
230250

231251
## Why `modify_allocation` returns a new buffer
232252

‎cuda_core/cuda/core/_memory/_buffer.pyi‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,11 @@
22

33
from typing import TypedDict
44

5-
from _typeshed import Incomplete
65
from cuda.core._memory._copy_enums import CopyOptions
6+
from cuda.core._memory._device_memory_resource import DeviceMemoryResource
77
from cuda.core._memory._ipc import IPCBufferDescriptor
8+
from cuda.core._memory._pinned_memory_resource import PinnedMemoryResource
9+
from cuda.core._memory._virtual_memory_resource import VirtualMemoryResource
810
from cuda.core._stream import Stream
911
from cuda.core._utils.pycompat import BufferProtocol
1012
from cuda.core.graph import GraphBuilder
@@ -81,7 +83,7 @@ class Buffer:
8183
:class:`Buffer` is closed or garbage collected.
8284
"""
8385
@classmethod
84-
def from_ipc_descriptor(cls, mr: Incomplete, ipc_descriptor: IPCBufferDescriptor, *, stream: Stream) -> Buffer:
86+
def from_ipc_descriptor(cls, mr: DeviceMemoryResource | PinnedMemoryResource | VirtualMemoryResource, ipc_descriptor: IPCBufferDescriptor, *, stream: Stream) -> Buffer:
8587
"""Import a buffer that was exported from another process.
8688
8789
Parameters

‎cuda_core/cuda/core/_memory/_buffer.pyx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ from cuda.core._dlpack import classify_dl_device, make_py_capsule
4444
from cuda.core._device import Device
4545

4646
if TYPE_CHECKING:
47+
from cuda.core._memory._virtual_memory_resource import VirtualMemoryResource
4748
from cuda.core.graph import GraphBuilder
4849

4950

‎cuda_core/cuda/core/_memory/_ipc.pyi‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,13 +64,21 @@ class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
6464
POSIX file descriptors the handles are :class:`IPCAllocationHandle`
6565
objects: the descriptor owns them and closes them when it is released, and
6666
``multiprocessing`` duplicates them into the receiving process. Plain
67-
``pickle`` cannot carry file descriptors.
67+
``pickle`` cannot carry file descriptors; send the descriptor, or the
68+
buffer, through ``multiprocessing`` (a ``Queue``, a ``Pipe``, ``Process``
69+
arguments, or a ``Pool``).
70+
71+
A live descriptor costs one file descriptor per allocation and keeps the
72+
physical memory allocated, in every process that holds a copy, until it is
73+
released. A descriptor placed on a ``Queue`` or sent to a ``Pool`` pins the
74+
memory in the sender until the receiver has unpickled it.
6875
6976
Note
7077
----
7178
The sizes are controlled by the exporting peer. Receivers must treat them
7279
as untrusted and import only through :meth:`Buffer.from_ipc_descriptor`,
73-
which fails instead of mapping a size the driver rejects.
80+
which checks what it can and fails instead of mapping a size the driver
81+
rejects.
7482
"""
7583
_handle_type: int
7684
_chunk_sizes: tuple[Any, ...]

‎cuda_core/cuda/core/_memory/_ipc.pyx‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -178,13 +178,21 @@ cdef class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
178178
POSIX file descriptors the handles are :class:`IPCAllocationHandle`
179179
objects: the descriptor owns them and closes them when it is released, and
180180
``multiprocessing`` duplicates them into the receiving process. Plain
181-
``pickle`` cannot carry file descriptors.
181+
``pickle`` cannot carry file descriptors; send the descriptor, or the
182+
buffer, through ``multiprocessing`` (a ``Queue``, a ``Pipe``, ``Process``
183+
arguments, or a ``Pool``).
184+
185+
A live descriptor costs one file descriptor per allocation and keeps the
186+
physical memory allocated, in every process that holds a copy, until it is
187+
released. A descriptor placed on a ``Queue`` or sent to a ``Pool`` pins the
188+
memory in the sender until the receiver has unpickled it.
182189
183190
Note
184191
----
185192
The sizes are controlled by the exporting peer. Receivers must treat them
186193
as untrusted and import only through :meth:`Buffer.from_ipc_descriptor`,
187-
which fails instead of mapping a size the driver rejects.
194+
which checks what it can and fails instead of mapping a size the driver
195+
rejects.
188196
"""
189197

190198
@staticmethod

‎cuda_core/cuda/core/_memory/_virtual_memory_resource.pyi‎

Lines changed: 37 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -99,12 +99,17 @@ class VirtualMemoryBuffer(Buffer):
9999
"""Descriptor for sharing this buffer with other processes.
100100
101101
The descriptor exports every physical allocation that backs the
102-
buffer as a shareable handle and is cached on the buffer. It requires
103-
a resource whose ``handle_type`` can be shared; see
102+
buffer as a shareable handle and is cached on the buffer until the
103+
buffer closes. Each exported allocation takes one file descriptor,
104+
and a live descriptor keeps the physical memory allocated in every
105+
process that holds a copy. It requires a resource whose
106+
``handle_type`` can be shared; see
104107
:attr:`VirtualMemoryResource.is_ipc_enabled`. Import it with
105108
:meth:`Buffer.from_ipc_descriptor` and a
106109
:class:`VirtualMemoryResource` of the receiving process, or send the
107-
buffer itself through ``multiprocessing``.
110+
buffer itself through ``multiprocessing``. A buffer imported from
111+
another process cannot be exported again; share the descriptor it
112+
was imported from.
108113
"""
109114
def close(self, stream: Stream | GraphBuilder | None=None) -> None:
110115
"""Release this buffer's share of its address range.
@@ -146,6 +151,11 @@ class VirtualMemoryResource(MemoryResource):
146151
how CUDA Virtual Memory Management works before using this. Other MemoryResource subclasses
147152
in cuda.core should already meet the common needs.
148153
154+
A shared buffer's descriptor comes from the exporting process and is not trusted:
155+
:meth:`Buffer.from_ipc_descriptor` checks what it can, and the driver rejects a size that
156+
does not match the exported allocation. Unpickling a shared buffer performs a live import,
157+
so unpickle buffers only from a trusted principal.
158+
149159
Notes
150160
-----
151161
Every buffer this resource returns is a :class:`VirtualMemoryBuffer` that
@@ -160,11 +170,15 @@ class VirtualMemoryResource(MemoryResource):
160170
161171
Buffers can be shared with other processes when ``config.handle_type``
162172
is ``"posix_fd"`` (Linux). :attr:`Buffer.ipc_descriptor` exports the
163-
physical allocations that back a buffer, :meth:`Buffer.from_ipc_descriptor`
164-
imports them with a ``VirtualMemoryResource`` of the receiving process,
165-
and a buffer sent through ``multiprocessing`` does both. The importing
166-
resource maps the shared memory for its own device with its own access
167-
options; see :attr:`is_ipc_enabled`.
173+
physical allocations that back a buffer, one file descriptor each;
174+
:meth:`Buffer.from_ipc_descriptor` imports them with a
175+
``VirtualMemoryResource`` of the receiving process for the device that
176+
owns the memory; and a buffer sent through ``multiprocessing`` does both.
177+
Plain ``pickle`` cannot carry the file descriptors. The importing resource
178+
maps the memory for its device and the devices in its ``peers`` option,
179+
with its own access options. A live descriptor keeps the memory allocated
180+
in every process that holds a copy. An imported buffer, and a buffer grown
181+
from one, cannot be exported again. See :attr:`is_ipc_enabled`.
168182
"""
169183
device: object
170184
config: object
@@ -210,7 +224,9 @@ class VirtualMemoryResource(MemoryResource):
210224
freed when the last of the two closes. When the driver can extend the
211225
address range in place, the returned buffer has the same pointer;
212226
otherwise it has a new one and the existing contents are reachable
213-
through both. Closing the returned buffer never closes ``buf``.
227+
through both. Closing the returned buffer never closes ``buf``. A
228+
buffer grown from an imported buffer maps imported memory and cannot
229+
be exported again.
214230
215231
Concurrent calls on buffers that alias one another are safe.
216232
@@ -224,7 +240,8 @@ class VirtualMemoryResource(MemoryResource):
224240
Configuration for the new physical memory chunk only. Existing
225241
chunks keep the access they were created with, and the resource's
226242
own configuration is unchanged. It must name the resource's
227-
``location_type`` and passes the same checks as the constructor.
243+
``location_type`` and ``handle_type`` and passes the same checks
244+
as the constructor.
228245
When ``buf`` already covers ``new_size`` there is no new chunk, so
229246
``config`` has no effect. This method never changes the access of
230247
memory that is already mapped.
@@ -241,8 +258,8 @@ class VirtualMemoryResource(MemoryResource):
241258
TypeError
242259
If ``buf`` did not come from this resource.
243260
ValueError
244-
If ``config`` names a different location than the resource, or
245-
the constructor would reject it.
261+
If ``config`` names a different location or handle type than the
262+
resource, or the constructor would reject it.
246263
RuntimeError
247264
If ``buf`` is closed, or ``config`` requests GPUDirect RDMA on a
248265
device without support.
@@ -297,10 +314,14 @@ class VirtualMemoryResource(MemoryResource):
297314
"""Import a buffer that another process exported; see :meth:`Buffer.from_ipc_descriptor`.
298315
299316
Every exported allocation is imported with this resource's handle
300-
type, then mapped in order into one new address reservation, with the
301-
access descriptors this resource's options produce for its device.
302-
The returned buffer owns the imported handles like a buffer from
303-
:meth:`allocate` owns created ones, and reports ``is_mapped``.
317+
type and checked to live where this resource allocates (its device,
318+
or its host location). The allocations are then mapped in order into
319+
one new address reservation with the access descriptors this
320+
resource's options produce for its device and peers, and the
321+
deallocation stream is recorded as :meth:`allocate` does. The
322+
returned buffer owns the imported handles like a buffer from
323+
:meth:`allocate` owns created ones, reports ``is_mapped``, and cannot
324+
be exported again.
304325
"""
305326
@property
306327
def device_id(self) -> int:

0 commit comments

Comments
 (0)