Skip to content

Commit 85f1d6d

Browse files
Andy-Jostclaude
andcommitted
cuda.core: VMM IPC review fixes: posix_fd-only from_fds, scoped docs, no test skips
- from_fds() takes no handle_type; a file descriptor can only carry posix_fd - The descriptor, resource, and release-note text scope the pin, pickle, and same-device rules to POSIX file descriptors, and the fds docstrings state the descriptor's lifetime rule - The 1.3.0 notes record that a per-call modify_allocation config must keep the resource's handle_type - VMM_DESIGN.md records the fabric direction: a descriptor class of its own - TestManyChunkBuffer and TestReexport no longer skip when the driver refuses the decoy's address hint; the grow relocates in that case too Co-Authored-By: Claude Fable 5.1 <[email protected]>
1 parent 2cf07d9 commit 85f1d6d

7 files changed

Lines changed: 144 additions & 126 deletions

File tree

‎cuda_core/cuda/core/_cpp/rt/VMM_DESIGN.md‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -144,8 +144,9 @@ what makes that safe: the allocation is released exactly once, when its last map
144144
`location_type="host"` with a handle type other than `None`, which the driver rejects, and a
145145
request for GPUDirect RDMA on a device without support. `__init__` runs it after the VMM-support
146146
check; `modify_allocation` runs it on a per-call configuration, which must also name the
147-
resource's location. The resource reports `is_ipc_enabled` (True for `posix_fd` on Linux with
148-
device-located memory), which `Buffer.ipc_descriptor` and `Buffer.from_ipc_descriptor` check.
147+
resource's location and `handle_type`. The resource reports `is_ipc_enabled` (True for
148+
`posix_fd` on Linux with device-located memory), which `Buffer.ipc_descriptor` and
149+
`Buffer.from_ipc_descriptor` check.
149150
- `cdef class VirtualMemoryBuffer(Buffer)` carries no extra state. It is created with
150151
`Buffer_from_deviceptr_handle(h_ptr, size, self, cls=VirtualMemoryBuffer)` and documented in
151152
`api.rst` like `ManagedBuffer`. It overrides `close(stream=None)` to reject a capturing stream
@@ -253,9 +254,9 @@ descriptor either; it records only that it was imported (`is_mapped`). A
253254
descriptor parked in a `Queue` or sent to a `Pool` pins the memory in the
254255
sender until the receiver unpickles it. Besides `multiprocessing`, a process
255256
with its own file descriptor passing sends the descriptor's `fds`, `sizes`,
256-
`handle_type`, and `size` and rebuilds it with
257-
`VirtualMemoryIPCBufferDescriptor.from_fds()`, which duplicates the integers
258-
it is given.
257+
and `size` and rebuilds it with `VirtualMemoryIPCBufferDescriptor.from_fds()`,
258+
which duplicates the integers it is given and always builds a `posix_fd`
259+
descriptor.
259260

260261
Re-export. The driver exports only allocations this process created with the
261262
requested handle type; an imported allocation has none, and the driver reports
@@ -291,6 +292,18 @@ stay opt-in.
291292
A `modify_allocation` config must keep the resource's `handle_type`: every
292293
chunk of a buffer must be exportable the same way.
293294

295+
Fabric handles. `CU_MEM_HANDLE_TYPE_FABRIC` exports a `CUmemFabricHandle`, 64
296+
bytes of plain data that travel through `pickle`, MPI, or a socket, need no
297+
close, and cannot hold a reference to the allocation while in flight. The
298+
importer needs access to the exporter's IMEX channel rather than a matching
299+
device index, and the memory may live on another node. The file-descriptor
300+
rules above (ownership, pinning, `pickle` raising, the same-device check) do
301+
not hold for it, so fabric sharing gets a descriptor class of its own instead
302+
of a widened `VirtualMemoryIPCBufferDescriptor`, whose `handles` stay
303+
`IPCAllocationHandle`s. `from_fds()` is POSIX-only by construction for the
304+
same reason and takes no handle type; a fabric descriptor is plain data and
305+
needs no out-of-band constructor. Tracked on the VMM epic (#2980).
306+
294307
## Why `modify_allocation` returns a new buffer
295308

296309
The input buffer stays open and aliases the result. The chunks the input already mapped are

‎cuda_core/cuda/core/_memory/_ipc.pyi‎

Lines changed: 25 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -61,12 +61,13 @@ class IPCAllocationHandle:
6161
class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
6262
"""Serializable object describing a :class:`VirtualMemoryBuffer` that can be shared between processes.
6363
64-
The descriptor holds one exported handle per physical allocation that
65-
backs the buffer, in address order (``handles``), with each allocation's
66-
size (``sizes``) and the buffer's size (``size``, which may be smaller
67-
than their sum). With POSIX file descriptors the handles are
68-
:class:`IPCAllocationHandle` objects: the descriptor owns them and closes
69-
them when it is released.
64+
The descriptor holds one exported POSIX file descriptor per physical
65+
allocation that backs the buffer, in address order (``handles``, each an
66+
:class:`IPCAllocationHandle` that the descriptor owns and closes when it
67+
is released), with each allocation's size (``sizes``) and the buffer's
68+
size (``size``, which may be smaller than their sum). ``handle_type`` is
69+
``"posix_fd"``, the only handle type cuda.core shares today; the rules
70+
below describe file descriptors.
7071
7172
A descriptor pins the physical memory while it exists, in every process
7273
that holds one, and nothing else does: the exporting buffer keeps no
@@ -81,8 +82,8 @@ class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
8182
``Process`` arguments, or a ``Pool``) pickles the descriptor, or a buffer,
8283
and duplicates the file descriptors into the receiving process; plain
8384
``pickle`` cannot carry them and raises. A process with its own file
84-
descriptor passing sends ``fds``, ``sizes``, ``handle_type``, and ``size``
85-
itself, and the receiver rebuilds the descriptor with :meth:`from_fds`.
85+
descriptor passing sends ``fds``, ``sizes``, and ``size`` itself, and the
86+
receiver rebuilds the descriptor with :meth:`from_fds`.
8687
8788
Note
8889
----
@@ -98,13 +99,14 @@ class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
9899
@staticmethod
99100
def _from_exports(handle_type: int, chunk_sizes: tuple[int, ...], handles: tuple[Any, ...], size: int) -> VirtualMemoryIPCBufferDescriptor: ...
100101
@classmethod
101-
def from_fds(cls, fds: Iterable[int | IPCAllocationHandle], sizes: Iterable[int], *, handle_type: VirtualMemoryHandleType | str='posix_fd', size: int | None=None) -> VirtualMemoryIPCBufferDescriptor:
102-
"""Build a descriptor from file descriptors received outside ``multiprocessing``.
102+
def from_fds(cls, fds: Iterable[int | IPCAllocationHandle], sizes: Iterable[int], *, size: int | None=None) -> VirtualMemoryIPCBufferDescriptor:
103+
"""Build a descriptor from POSIX file descriptors received outside ``multiprocessing``.
103104
104-
The exporter sends ``fds``, ``sizes``, ``handle_type``, and ``size``
105-
of its descriptor through a transport of its own (a Unix socket with
106-
``SCM_RIGHTS``, for example); the receiver rebuilds the descriptor
107-
here and imports it with :meth:`Buffer.from_ipc_descriptor`.
105+
The exporter sends ``fds``, ``sizes``, and ``size`` of its descriptor
106+
through a transport of its own (a Unix socket with ``SCM_RIGHTS``, for
107+
example); the receiver rebuilds the descriptor here and imports it
108+
with :meth:`Buffer.from_ipc_descriptor`. The result's ``handle_type``
109+
is ``"posix_fd"``, the only handle type a file descriptor can carry.
108110
109111
Parameters
110112
----------
@@ -116,17 +118,15 @@ class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
116118
holder.
117119
sizes : Iterable[int]
118120
The exporter's ``sizes``, one per handle.
119-
handle_type : VirtualMemoryHandleType | str, optional
120-
The exporter's ``handle_type``; ``"posix_fd"`` by default.
121121
size : int, optional
122122
The exporter's ``size``, at most the sum of ``sizes``; the sum by
123123
default.
124124
125125
Raises
126126
------
127127
ValueError
128-
If the counts differ, a size is not positive, ``size`` exceeds the
129-
sum of ``sizes``, or ``handle_type`` cannot be shared.
128+
If the counts differ, a size is not positive, or ``size`` exceeds
129+
the sum of ``sizes``.
130130
RuntimeError
131131
If an :class:`IPCAllocationHandle` is closed.
132132
OSError
@@ -140,13 +140,18 @@ class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
140140
"""Size of each physical allocation, in address order; the sum is at least ``size``."""
141141
@property
142142
def handles(self) -> tuple[IPCAllocationHandle, ...]:
143-
"""The exported handles, one per allocation, in address order; the descriptor owns them."""
143+
"""The exported handles, one :class:`IPCAllocationHandle` per allocation, in address order.
144+
145+
The descriptor owns them and closes them when it is released.
146+
"""
144147
@property
145148
def fds(self) -> tuple[int, ...]:
146149
"""The exported file descriptors as integers, for a transport of your own.
147150
148151
They belong to this descriptor: send or duplicate them while it is
149-
alive. Each one is released when the descriptor is.
152+
alive, because each one is closed when the descriptor is released.
153+
Keep the descriptor in a variable while you use them; the numbers
154+
from ``buf.ipc_descriptor.fds`` alone may already be closed.
150155
"""
151156
def __reduce__(self) -> tuple[object, ...]: ...
152157

‎cuda_core/cuda/core/_memory/_ipc.pyx‎

Lines changed: 25 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -180,12 +180,13 @@ multiprocessing.reduction.register(IPCAllocationHandle, _reduce_allocation_handl
180180
cdef class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
181181
"""Serializable object describing a :class:`VirtualMemoryBuffer` that can be shared between processes.
182182
183-
The descriptor holds one exported handle per physical allocation that
184-
backs the buffer, in address order (``handles``), with each allocation's
185-
size (``sizes``) and the buffer's size (``size``, which may be smaller
186-
than their sum). With POSIX file descriptors the handles are
187-
:class:`IPCAllocationHandle` objects: the descriptor owns them and closes
188-
them when it is released.
183+
The descriptor holds one exported POSIX file descriptor per physical
184+
allocation that backs the buffer, in address order (``handles``, each an
185+
:class:`IPCAllocationHandle` that the descriptor owns and closes when it
186+
is released), with each allocation's size (``sizes``) and the buffer's
187+
size (``size``, which may be smaller than their sum). ``handle_type`` is
188+
``"posix_fd"``, the only handle type cuda.core shares today; the rules
189+
below describe file descriptors.
189190
190191
A descriptor pins the physical memory while it exists, in every process
191192
that holds one, and nothing else does: the exporting buffer keeps no
@@ -200,8 +201,8 @@ cdef class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
200201
``Process`` arguments, or a ``Pool``) pickles the descriptor, or a buffer,
201202
and duplicates the file descriptors into the receiving process; plain
202203
``pickle`` cannot carry them and raises. A process with its own file
203-
descriptor passing sends ``fds``, ``sizes``, ``handle_type``, and ``size``
204-
itself, and the receiver rebuilds the descriptor with :meth:`from_fds`.
204+
descriptor passing sends ``fds``, ``sizes``, and ``size`` itself, and the
205+
receiver rebuilds the descriptor with :meth:`from_fds`.
205206
206207
Note
207208
----
@@ -230,15 +231,15 @@ cdef class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
230231
fds: Iterable[int | IPCAllocationHandle],
231232
sizes: Iterable[int],
232233
*,
233-
handle_type: VirtualMemoryHandleType | str = "posix_fd",
234234
size: int | None = None,
235235
) -> VirtualMemoryIPCBufferDescriptor:
236-
"""Build a descriptor from file descriptors received outside ``multiprocessing``.
236+
"""Build a descriptor from POSIX file descriptors received outside ``multiprocessing``.
237237

238-
The exporter sends ``fds``, ``sizes``, ``handle_type``, and ``size``
239-
of its descriptor through a transport of its own (a Unix socket with
240-
``SCM_RIGHTS``, for example); the receiver rebuilds the descriptor
241-
here and imports it with :meth:`Buffer.from_ipc_descriptor`.
238+
The exporter sends ``fds``, ``sizes``, and ``size`` of its descriptor
239+
through a transport of its own (a Unix socket with ``SCM_RIGHTS``, for
240+
example); the receiver rebuilds the descriptor here and imports it
241+
with :meth:`Buffer.from_ipc_descriptor`. The result's ``handle_type``
242+
is ``"posix_fd"``, the only handle type a file descriptor can carry.
242243

243244
Parameters
244245
----------
@@ -250,29 +251,21 @@ cdef class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
250251
holder.
251252
sizes : Iterable[int]
252253
The exporter's ``sizes``, one per handle.
253-
handle_type : VirtualMemoryHandleType | str, optional
254-
The exporter's ``handle_type``; ``"posix_fd"`` by default.
255254
size : int, optional
256255
The exporter's ``size``, at most the sum of ``sizes``; the sum by
257256
default.
258257

259258
Raises
260259
------
261260
ValueError
262-
If the counts differ, a size is not positive, ``size`` exceeds the
263-
sum of ``sizes``, or ``handle_type`` cannot be shared.
261+
If the counts differ, a size is not positive, or ``size`` exceeds
262+
the sum of ``sizes``.
264263
RuntimeError
265264
If an :class:`IPCAllocationHandle` is closed.
266265
OSError
267266
If an ``int`` is not an open file descriptor.
268267
"""
269-
# Lazy import: this module is loaded while _virtual_memory_resource imports it.
270-
from cuda.core._memory._virtual_memory_resource import VirtualMemoryResourceOptions
271-
from cuda.core.typing import VirtualMemoryHandleType
272-
273-
if handle_type is None:
274-
raise ValueError("handle_type=None cannot be shared; the exporter's handle_type is required")
275-
driver_type = int(VirtualMemoryResourceOptions._handle_type_to_driver(VirtualMemoryHandleType(handle_type)))
268+
cdef int driver_type = <int>cydriver.CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR
276269
cdef tuple sizes_t = tuple(operator.index(n) for n in sizes)
277270
cdef list given = list(fds)
278271
if len(given) != len(sizes_t):
@@ -318,15 +311,20 @@ cdef class VirtualMemoryIPCBufferDescriptor(IPCBufferDescriptor):
318311

319312
@property
320313
def handles(self) -> tuple[IPCAllocationHandle, ...]:
321-
"""The exported handles, one per allocation, in address order; the descriptor owns them."""
314+
"""The exported handles, one :class:`IPCAllocationHandle` per allocation, in address order.
315+
316+
The descriptor owns them and closes them when it is released.
317+
"""
322318
return self._handles
323319

324320
@property
325321
def fds(self) -> tuple[int, ...]:
326322
"""The exported file descriptors as integers, for a transport of your own.
327323

328324
They belong to this descriptor: send or duplicate them while it is
329-
alive. Each one is released when the descriptor is.
325+
alive, because each one is closed when the descriptor is released.
326+
Keep the descriptor in a variable while you use them; the numbers
327+
from ``buf.ipc_descriptor.fds`` alone may already be closed.
330328
"""
331329
return tuple(int(handle) for handle in self._handles)
332330

‎cuda_core/cuda/core/_memory/_virtual_memory_resource.pyi‎

Lines changed: 24 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -102,9 +102,12 @@ class VirtualMemoryBuffer(Buffer):
102102
Every access exports the physical allocations that back the buffer
103103
again and returns a new descriptor that owns one file descriptor per
104104
allocation; the file descriptors close when the descriptor is
105-
released. A descriptor pins the physical memory while it exists, in
106-
every process that holds one, and nothing else does: the buffer
107-
itself keeps no descriptor, so it pins nothing beyond its own mapping.
105+
released. Keep the descriptor in a variable while you use its
106+
``fds``: the numbers from ``buf.ipc_descriptor.fds`` alone may
107+
already be closed. A descriptor pins the physical memory while it
108+
exists, in every process that holds one, and nothing else does: the
109+
buffer itself keeps no descriptor, so it pins nothing beyond its own
110+
mapping.
108111
Pickling the buffer through ``multiprocessing`` builds a transient
109112
descriptor, so the sender holds no file descriptors between sends;
110113
a buffer passed as a ``Process`` argument is the exception, because
@@ -186,23 +189,24 @@ class VirtualMemoryResource(MemoryResource):
186189
:meth:`Buffer.set_deallocation_stream`.
187190
188191
Buffers can be shared with other processes when ``config.handle_type``
189-
is ``"posix_fd"`` (Linux); ``handle_type`` is the switch and
190-
:attr:`is_ipc_enabled` the query. :attr:`Buffer.ipc_descriptor` exports
191-
the physical allocations that back a buffer, one file descriptor each,
192-
into a new descriptor on every access; :meth:`Buffer.from_ipc_descriptor`
193-
imports a descriptor with a ``VirtualMemoryResource`` of the receiving
194-
process for the device that owns the memory; and a buffer sent through
195-
``multiprocessing`` does both. Plain ``pickle`` cannot carry the file
196-
descriptors; a process with its own file descriptor passing sends the
197-
descriptor's ``fds``, ``sizes``, ``handle_type``, and ``size`` and
198-
rebuilds it with ``VirtualMemoryIPCBufferDescriptor.from_fds()``. The
199-
importing resource maps the memory for its device and the devices in its
200-
``peers`` option, with its own access options. A descriptor pins the
201-
physical memory while it exists, in every process that holds one, and
202-
nothing else does. A buffer that contains imported memory (an import, or
203-
anything :meth:`modify_allocation` derived from one) cannot be exported
204-
again; forward the descriptor it came from, or copy into a buffer you
205-
own.
192+
is ``"posix_fd"`` (Linux), the only handle type cuda.core shares today;
193+
``handle_type`` is the switch and :attr:`is_ipc_enabled` the query. The
194+
rules that follow describe POSIX file descriptors.
195+
:attr:`Buffer.ipc_descriptor` exports the physical allocations that back
196+
a buffer, one file descriptor each, into a new descriptor on every
197+
access; :meth:`Buffer.from_ipc_descriptor` imports a descriptor with a
198+
``VirtualMemoryResource`` of the receiving process for the device that
199+
owns the memory; and a buffer sent through ``multiprocessing`` does both.
200+
Plain ``pickle`` cannot carry the file descriptors; a process with its
201+
own file descriptor passing sends the descriptor's ``fds``, ``sizes``,
202+
and ``size`` and rebuilds it with
203+
``VirtualMemoryIPCBufferDescriptor.from_fds()``. The importing resource
204+
maps the memory for its device and the devices in its ``peers`` option,
205+
with its own access options. A descriptor pins the physical memory while
206+
it exists, in every process that holds one, and nothing else does. A
207+
buffer that contains imported memory (an import, or anything
208+
:meth:`modify_allocation` derived from one) cannot be exported again;
209+
forward the descriptor it came from, or copy into a buffer you own.
206210
"""
207211
device: object
208212
config: object

0 commit comments

Comments
 (0)