Isaac Sim Version
4.5.0
Operating System
Ubuntu 22.04
GPU Information
- Model: RTX A6000
- Driver Version: 570.86.10
Topic Description
I get a GPU crash from the isaac-tiamat/GroundControl repo, which Nvidia engineers have access to.
Detailed Description
Steps to Reproduce
- Follow the setup instructions at https://gitlab.sitcore.net/tiamat/isaac-tiamat/README.md
- python GroundControl/scripts/environments/teleoperation/teleop_se2_agent.py --task Isaac-Navigation-Flat-Spot-Play-v0 --num_envs 1 --teleop_device keyboard
Error Messages
IsaacSim is stuck loading at 50%. The console output contains errors as follows:
2025-04-28 20:36:35 [52,606ms] [Warning] [rtx.rtxsensor.plugin] Using User-specified coordinate frame quaternion (0.000000, 0.000000, 0.000000, 1.000000) in RTXSensor
Warning: Possible version incompatibility. Attempting to load omni::fabric::IPrimBucketList with version v0.4 against v0.3.
2025-04-28 20:36:36 [52,930ms] [Warning] [carb.graphics-vulkan.plugin] GPU pagefault occured on virtual address(0x0000004d00000000)
2025-04-28 20:36:36 [52,930ms] [Warning] [carb.graphics-vulkan.plugin] GPU pagefault no associated data found closest allocations were:
2025-04-28 20:36:36 [52,931ms] [Warning] [carb.graphics-vulkan.plugin] 1: AddressRange(0x0000000000000000 - 0x0000000000000000) with name(dummy to ignore)
2025-04-28 20:36:36 [52,931ms] [Warning] [carb.graphics-vulkan.plugin] 2: AddressRange(0xffffffffffffffff - 0xffffffffffffffff) with name(dummy to ignore)
2025-04-28 20:36:36 [52,931ms] [Warning] [carb.graphics-vulkan.plugin] failed to find shader info with aftermath hash 0
2025-04-28 20:36:36 [52,931ms] [Warning] [carb.graphics-vulkan.plugin] failed to find shader info with aftermath hash 0
2025-04-28 20:36:36 [52,931ms] [Error] [carb.graphics-vulkan.plugin] GPU crash is detected. Trying to write crash dump into: /homes/elenius/miniconda3/envs/tiamat/lib/python3.10/site-packages/omni/logs/Kit/Isaac-Sim/4.5/kit_20250428_133543-0.nv-gpudmp
2025-04-28 20:36:36 [52,931ms] [Error] [carb.graphics-vulkan.plugin] GPU crash dump is successfully written
2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] GPU pagefault occured on virtual address(0x0000004d00000000)
2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] GPU pagefault no associated data found closest allocations were:
2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] 1: AddressRange(0x0000000000000000 - 0x0000000000000000) with name(dummy to ignore)
2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] 2: AddressRange(0xffffffffffffffff - 0xffffffffffffffff) with name(dummy to ignore)
2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] failed to find shader info with aftermath hash 0
2025-04-28 20:36:36 [52,936ms] [Warning] [carb.graphics-vulkan.plugin] failed to find shader info with aftermath hash 0
2025-04-28 20:36:36 [52,936ms] [Error] [carb.graphics-vulkan.plugin] GPU crash is detected. Trying to write crash dump into: /homes/elenius/miniconda3/envs/tiamat/lib/python3.10/site-packages/omni/logs/Kit/Isaac-Sim/4.5/kit_20250428_133543-1.nv-gpudmp
2025-04-28 20:36:36 [52,936ms] [Error] [carb.graphics-vulkan.plugin] GPU crash dump is successfully written
zip file with the two .nv-gpudmp files mentioned above attached.
gpudmp.zip (18.0 KB)