Issue with NVIDIA NIM on RTX 5090 Ubuntu 24.04 Server

AI & Data Science NVIDIA NIM NIM on RTX AI PCs and Workstations
, ,
1

I have installed the NVIDIA 580 driver; results:

±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.82.07 Driver Version: 580.82.07 CUDA Version: 13.0 |
±----------------------------------------±-----------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 5090 Off | 00000000:01:00.0 Off | N/A |
| 0% 28C P8 22W / 600W | 83MiB / 32607MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+

±----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
±----------------------------------------------------------------------------------------+

CUDA installed as well:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Aug_20_01:58:59_PM_PDT_2025
Cuda compilation tools, release 13.0, V13.0.88
Build cuda_13.0.r13.0/compiler.36424714_0

Installed the NVIDIA Container Toolkit 1.17.8 as per the steps from NVIDIA Docs (here)

When I tried to run a test NIM Sample:

sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

I got the error:

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: failed to fulfil mount request: open /run/nvidia-persistenced/socket: no such file or directory: unknown

Any clues what is wrong and how to resolve the issue?

2

Hi Samimust,

We were not able to reproduce your issue.

Looks like issues with docker not able to start the container properly. Usually happens if path is not properly configured, or user has permission issues.

Can you try uninstalling nvidia driver, reinstall driver, CTK and container tool kit, also confirm docker works without issues (https://docs.docker.com/engine/install/ubuntu/) and then try?

To uninstall nvidia driver :

sudo add-apt-repository ppa:graphics-drivers/ppa

sudo apt update

apt list --installed | grep nvidia

apt remove nvidia-driver-580-open

apt remove nvidia-driver-580

apt autoremove

Thanks!