Skip to content

Support viewing CuPy arrays as cuda::std::mdspan on host/device - #9525

Merged
kmaehashi merged 13 commits into
cupy:mainfrom
leofang:mdspan_like
Jan 27, 2026
Merged

Support viewing CuPy arrays as cuda::std::mdspan on host/device#9525
kmaehashi merged 13 commits into
cupy:mainfrom
leofang:mdspan_like

Conversation

@leofang

@leofang leofang commented Dec 16, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

This PR introduces a way for CuPy to interoperate with CUDA C++, by supporting emitting cuda::std::mdspan on the host that can be passed by value to the device. Same as how we have been treating CArray over the years, we exploit the cuda::std::mdspan ABI stability to allow dynamically generating the struct.

With this PR, it really helps reduce overhead of generating kernel args (compared to e.g. DLPack which has O(1 us) overhead):

In [1]: import cupy as cp

In [2]: arr = cp.arange(32).reshape(4, 8)

In [3]: %timeit mdspan = arr.mdspan(index_type=cp.int32)
199 ns ± 4.23 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

xref: NVIDIA/cuda-python#1387

@leofang leofang added this to the v14 milestone Dec 16, 2025
@leofang leofang mentioned this pull request Dec 16, 2025
Draft
2 tasks
@mergify

mergify Bot commented Dec 22, 2025

Copy link
Copy Markdown
Contributor

This pull request is now in conflicts. Could you fix it @leofang? 🙏

@leofang leofang modified the milestones: v14, v14.0.0, v15 Dec 24, 2025
@leofang leofang added the to-be-backported Pull-requests to be backported to stable branch label Dec 29, 2025
@leofang leofang added the blocking Issue/pull-request is mandatory for the upcoming release label Jan 20, 2026
mhoemmen
mhoemmen suggested changes Jan 21, 2026
View reviewed changes

@mhoemmen mhoemmen left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the great discussion! I recommend either using layout_stride_relaxed in this PR, or restricting the input to have nonnegative strides and using the recommendations in P3959.

Comment thread cupy/_core/core.pyx Resolved
Comment thread cupy/_core/_carray.pyx Resolved
Comment thread cupy/_core/_carray.pyx Resolved
Comment thread cupy/_core/_carray.pyx Resolved
Comment thread cupy/_core/_carray.pyx Resolved
Comment thread tests/cupy_tests/core_tests/test_mdspan.py Resolved
@leofang leofang added the cat:feature New features/APIs label Jan 21, 2026
@leofang

leofang commented Jan 21, 2026

Copy link
Copy Markdown
Member Author

Thanks to Mark for offline discussion. This PR is ready for review. I'll expand the PR description tonight/tomorrow.

@leofang leofang changed the title WIP: Support viewing CuPy arrays as cuda::std::mdspan on host/device Support viewing CuPy arrays as cuda::std::mdspan on host/device Jan 21, 2026
@leofang
leofang marked this pull request as ready for review January 21, 2026 23:49
@leofang
leofang requested a review from a team as a code owner January 21, 2026 23:49
seberg
seberg reviewed Jan 22, 2026
View reviewed changes

@seberg seberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I lost a bit focus on the tests, but this looks good to me. Adding mdspan seems fine. And I think I'll brush away my API concerns, by considering that we can add a layout="auto" in some future (and if we care a lot, we can probably pull off transitioning to it as the default too).

A few small comments, nits otherwise.

Comment thread tests/cupy_tests/core_tests/test_mdspan.py
name_expressions=name_expressions
)

# TODO(leofang): it does not seem 'AK' makes any difference here?

@seberg seberg Jan 21, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You would need to transpose a 3-D array and shaped_random has no logic to intentionally mangle it up.
I.e. if you truly want this, you need to test 3-D and then parametrize a transpose with (0, 1, 2), (2, 1, 0), (2, 0, 1) where the last would be neither C nor F.

(There is a helper for "contiguous axes", but I am not sure there is one for this.)

Comment thread cupy/_core/_carray.pyx Resolved
Comment thread cupy/_core/core.pyx Resolved
Comment thread cupy/_core/_carray.pyx Resolved
Comment thread cupy/_core/_carray.pyx Resolved
Comment thread cupy/_core/core.pyx Resolved
Comment thread cupy/_core/core.pyx Resolved
Comment thread cupy/_core/core.pyx Outdated
# would cause cyclic cimport dependencies.
cdef _carray.mdspan carr = _carray.mdspan.__new__(_carray.mdspan)
cdef int index_itemsize
if index_type == cupy.int32:

@seberg seberg Jan 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The speed discussion (from elsewhere) made me realize that technically, this should use get_type(index_type) == cupy.int32 maybe? Then we also allow writing "int32", but mainly we can't run into problems because np.int != np.long even when they are both 32bit.

Comment thread cupy/cuda/function.pxd Resolved
Comment thread tests/cupy_tests/core_tests/test_mdspan.py Resolved
mhoemmen
mhoemmen reviewed Jan 22, 2026
View reviewed changes
Comment thread cupy/_core/_scalar.pyx Resolved
@leofang leofang mentioned this pull request Jan 22, 2026
Open
@leofang

leofang commented Jan 22, 2026

Copy link
Copy Markdown
Member Author

/test mini

@seberg

seberg commented Jan 23, 2026

Copy link
Copy Markdown
Member

(timeouts here as well)

/test mini

seberg
seberg previously approved these changes Jan 23, 2026
View reviewed changes

@seberg seberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I am happy with the latest changes and much more comfortable with forcing to pass the index type!

@leofang

leofang commented Jan 24, 2026

Copy link
Copy Markdown
Member Author

Fixed test failures due to changing CPointer.ptr from void* to intptr_t.

/test mini

@kmaehashi

kmaehashi commented Jan 27, 2026

Copy link
Copy Markdown
Member

LGTM. Ignoring timeouts.
/test skip

@kmaehashi
kmaehashi enabled auto-merge January 27, 2026 06:52
@seberg

seberg commented Jan 27, 2026

Copy link
Copy Markdown
Member

/test force-skip

@github-actions

github-actions Bot commented Jan 27, 2026

Copy link
Copy Markdown

The following tests were force-skipped:

@kmaehashi
kmaehashi merged commit 8ccca69 into cupy:main Jan 27, 2026
51 of 52 checks passed
chainer-ci pushed a commit to chainer-ci/cupy that referenced this pull request Jan 27, 2026
@kmaehashi @chainer-ci
Merge pull request cupy#9525 from leofang/mdspan_like
9b33752 
Support viewing CuPy arrays as `cuda::std::mdspan` on host/device
@chainer-ci chainer-ci mentioned this pull request Jan 27, 2026
Merged
@leofang
leofang deleted the mdspan_like branch January 28, 2026 06:07
@kmaehashi kmaehashi mentioned this pull request Feb 17, 2026
Merged
@leofang leofang modified the milestones: v15, v15.0.0a1 Mar 7, 2026
@kmaehashi kmaehashi mentioned this pull request May 30, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kmaehashi kmaehashi kmaehashi approved these changes
@seberg seberg seberg left review comments
+1 more reviewer
@mhoemmen mhoemmen mhoemmen requested changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

blocking Issue/pull-request is mandatory for the upcoming release cat:feature New features/APIs to-be-backported Pull-requests to be backported to stable branch

Projects

None yet

Milestone

v15.0.0a1

Development

Successfully merging this pull request may close these issues.

4 participants

@leofang @seberg @kmaehashi @mhoemmen