Skip to content

Commit 13ab2f1

Browse files
Andy-Jostclaude
andcommitted
cuda.core: hold ipc_import_mutex while freeing an imported IPC pointer
With the GIL reorder in place, the concurrent-import test crashes on CUDA 12.9 (SIGSEGV, or glibc "double free or corruption") because the deleter did not take ipc_import_mutex. When the last reference to a cached import drops, the cache entry's weak_ptr expires before the deleter has freed the mapping. A concurrent importer that looks the key up in that window sees a miss and imports the allocation again; the driver returns a duplicate pointer to the same mapping (nvbug 5570902), and the first cuMemFreeAsync unmaps it for both. Release the GIL first, then hold the mutex across unregister and free, so the concurrent importer waits until the mapping is gone and imports fresh. Co-Authored-By: Claude Fable 5.1 <[email protected]>
1 parent 7d2819d commit 13ab2f1

1 file changed

Lines changed: 7 additions & 1 deletion

File tree

‎cuda_core/cuda/core/_cpp/resource_handles.cpp‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1545,8 +1545,14 @@ DevicePtrHandle deviceptr_import_ipc(const MemoryPoolHandle& h_pool, const void*
15451545
auto box = std::shared_ptr<DevicePtrBox>(
15461546
new DevicePtrBox{ptr, std::move(ds)},
15471547
[h_pool, key](DevicePtrBox* b) {
1548-
ipc_ptr_cache.unregister_handle(key);
1548+
// Release the GIL first (the GIL is the outermost lock), then hold
1549+
// the mutex across unregister + free. A concurrent import that finds
1550+
// this entry expired must wait until the mapping is gone; otherwise
1551+
// it re-imports the same allocation and the first cuMemFreeAsync
1552+
// unmaps it for both (nvbug 5570902).
15491553
GILReleaseGuard gil;
1554+
std::lock_guard<std::mutex> lock(ipc_import_mutex);
1555+
ipc_ptr_cache.unregister_handle(key);
15501556
const DeallocationStream& stream = b->deallocation;
15511557
cleanup_in_context(
15521558
deallocation_context(stream), "cuMemFreeAsync",

0 commit comments

Comments
 (0)