Implementing the Python Buffer Protocol

The Python Buffer Protocol explains how data can be viewed in different forms without copying. For example, memmap implements this protocol and so does NumPy's ndarray, which makes it easy to use a NumPy ndarray with a large on disk binary blob without accidentally copying it all into memory. In the future this could be a very important piece of Python's proposed Pickle Protocol 5, which would allow for zero-copy transmission between Python processes.

This could be a useful feature for CuPy's ndarrays. For example, this could make a Cython Interface ( #130 ) more performant as one can then assign CuPy's ndarrays to Cython typed memoryviews without copying. OTOH as memory for CuPy arrays likely lives in GPU memory (though I don't know the internals well), this could lead to some weird cases as well (e.g. using a NumPy array to interface with data in GPU memory).

Anyways figured I'd open this for discussion at this point. Thoughts welcome.

ref: https://docs.python.org/3/c-api/buffer.html
ref: https://www.python.org/dev/peps/pep-3118/
ref: https://www.python.org/dev/peps/pep-0574/

@jakirkham

@jakirkham In my opinion, while supporting the buffer protocol should be easy (see Refs 1, 2), it has little use in practice for GPU computation.

The reason is that both the producer and the consumer of the buffer live on the host while the actual buffer lives on the device. Even if you just want a read-only view of a cupy.ndarray, the moment the consumer parses a view the program will just crash as there's no way to call CUDA APIs for data transfer when executing the protocol.

I think __cuda_array_interface__ should cover your need well.

[1] https://cython.readthedocs.io/en/latest/src/userguide/buffer.html
[2] Note that I didn't take care of the format field:

diff --git a/cupy/core/core.pyx b/cupy/core/core.pyx
index 8c03767..e7f67aa 100644
--- a/cupy/core/core.pyx
+++ b/cupy/core/core.pyx
@@ -151,6 +151,22 @@ cdef class ndarray:

         return desc

+    def __getbuffer__(self, Py_buffer *buffer, int flags):
+        buffer.buf = <void*>self.data.ptr
+        buffer.format = 'f'                     # float
+        buffer.internal = NULL                  # see References
+        buffer.itemsize = self.dtype.itemsize
+        buffer.len = self.size * self.dtype.itemsize   # product(shape) * itemsize
+        buffer.ndim = self.ndim
+        buffer.obj = self
+        buffer.readonly = 0
+        buffer.shape = self._shape.data()
+        buffer.strides = self._strides.data()
+        buffer.suboffsets = NULL                # for pointer arrays only
+
+    def __releasebuffer__(self, Py_buffer *buffer):
+        pass
+
     # The definition order of attributes and methods are borrowed from the
     # order of documentation at the following NumPy document.
     # https://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html

Test:

import numpy as np
import cupy as cp
a = cp.random.random((10,), dtype=cp.float32)
b = np.array(a)

Even if you just want a read-only view of a cupy.ndarray, the moment the consumer parses a view the program will just crash as there's no way to call CUDA APIs for data transfer when executing the protocol.

Well, I was just thinking about this and decided to take a quick look. Turns out in this case when performing b = np.array(a, dtype=cp.float32) there's an internal copy causing the segfault:

(gdb) bt
#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:225
#1  0x00007fa6149c7174 in _contig_to_contig ()
   from /home/leofang/miniconda3/envs/cupy_rocm35_dev/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so
#2  0x00007fa614953dc9 in raw_array_assign_array ()
   from /home/leofang/miniconda3/envs/cupy_rocm35_dev/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so
#3  0x00007fa614954547 in PyArray_AssignArray ()
   from /home/leofang/miniconda3/envs/cupy_rocm35_dev/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so
#4  0x00007fa614966a37 in PyArray_FromArray ()
   from /home/leofang/miniconda3/envs/cupy_rocm35_dev/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so
#5  0x00007fa614969c79 in PyArray_FromAny ()
   from /home/leofang/miniconda3/envs/cupy_rocm35_dev/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so
#6  0x00007fa614969dbb in PyArray_CheckFromAny ()
   from /home/leofang/miniconda3/envs/cupy_rocm35_dev/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so
#7  0x00007fa6149faf2c in _array_fromobject ()
   from /home/leofang/miniconda3/envs/cupy_rocm35_dev/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so
#8  0x000055eb3dc4d28c in _PyMethodDef_RawFastCallKeywords () at /home/conda/feedstock_root/build_artifacts/python_1596159872474/work/Objects/call.c:693
#9  0x000055eb3dc834b0 in _PyCFunction_FastCallKeywords (kwnames=<optimized out>, nargs=<optimized out>, args=0x7fa6160bb5c0, func=0x7fa614c6c8c0)
    at /home/conda/feedstock_root/build_artifacts/python_1596159872474/work/Objects/call.c:732
#10 call_function () at /home/conda/feedstock_root/build_artifacts/python_1596159872474/work/Python/ceval.c:4568
#11 0x000055eb3dcc7aee in _PyEval_EvalFrameDefault () at /home/conda/feedstock_root/build_artifacts/python_1596159872474/work/Python/ceval.c:3139
#12 0x000055eb3dc1cdb2 in PyEval_EvalFrameEx (throwflag=0, f=0x7fa6160bb450)
    at /home/conda/feedstock_root/build_artifacts/python_1596159872474/work/Python/ceval.c:547
#13 _PyEval_EvalCodeWithName () at /home/conda/feedstock_root/build_artifacts/python_1596159872474/work/Python/ceval.c:3930
#14 0x000055eb3dc1e1e3 in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, kws=0x0, argcount=0, args=0x0, locals=<optimized out>, 
    globals=<optimized out>, _co=<optimized out>) at /home/conda/feedstock_root/build_artifacts/python_1596159872474/work/Python/ceval.c:3959
#15 PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>)
...

Perhaps it's just me using the buffer protocol with NumPy incorrectly, but it also shows how unsafe it can be...

The good news is it seems Cython does most of the work for us. Taking the view is working: mem = memoryview(a).

Closing. I implemented this as part of the Grace Hopper support (#8442), but currently it is only enabled if on Grace Hopper (docs).

Navigation Menu

Repository navigation

Implementing the Python Buffer Protocol #1532

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Implementing the Python Buffer Protocol #1532

Description

Activity

leofang commented on Oct 3, 2019

leofang commented on Mar 8, 2021

leofang commented on Oct 31, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions