Repository navigation
[FIX] Use the opt-in shared memory limit in GetKernelWithMaxDynamicSharedMemory - #973
Merged
Merged
Conversation
…aredMemory `GetKernelWithMaxDynamicSharedMemory(name, -1)` is documented to allow the maximum available dynamic shared memory (device max minus the kernel's static shared memory). It computed that from device attribute 8 (`MaxSharedMemoryPerBlock`), the 48 KiB default per-block limit, so the attribute it set was what a kernel can already use without opting in, and launches with more than 48 KiB of dynamic shared memory still failed with an invalid-argument error. Query attribute 97 (`MaxSharedMemoryPerBlockOptin` / `CU_DEVICE_ATTRIBUTE_MAX_SHARED_MEMORY_PER_BLOCK_OPTIN`) instead. An explicit `dynamic_smem_max` is unchanged. Signed-off-by: 0z5a <[email protected]>
Signed-off-by: tqchen <[email protected]>
Signed-off-by: tqchen <[email protected]>
Signed-off-by: tqchen <[email protected]>
tqchen
approved these changes
Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #971
GetKernelWithMaxDynamicSharedMemory(name, -1)should allow the maximum dynamic shared memory the device permits, minus the kernel's static shared memory. It computed that limit from device attribute 8 (MaxSharedMemoryPerBlock), the 48 KiB default per-block limit. The attribute it set was therefore what a kernel can already use without opting in, and launches with more than 48 KiB of dynamic shared memory still failed withinvalid argument.This change queries attribute 97 (
cudaDevAttrMaxSharedMemoryPerBlockOptin/CU_DEVICE_ATTRIBUTE_MAX_SHARED_MEMORY_PER_BLOCK_OPTIN) instead. An explicitdynamic_smem_maxand launches up to 48 KiB behave as before.The new
test_cubin_launcher_max_dynamic_smemloads a kernel with 1 KiB of static shared memory throughGetKernelWithMaxDynamicSharedMemory(name, -1). It checks thatmaxDynamicSharedSizeBytes == optin - static, then launches with that much dynamic shared memory and checks the result. On GH200 (aarch64, sm_90) and B300 (sm_103) it fails on main (maxDynamicSharedSizeBytes=48128, expected 231424) and passes with this change. The three existing CUBIN tests still pass on both.The new test is behind the existing
torch.version.cuda > (13, 0)gate, like the other CUBIN tests. To run it with a cu130 torch ("13.0"), the gate was relaxed locally; that change is not part of this PR.