Hi, at R38.2.1 version, i found tegrastats always show gpu utlilization as 0%, please help check it.
nvidia-smi can show gpu utlization.
i aslo checked orin, tegrastats can show gpu utlization.
but nvidia-smi pmon can’t show sm utlization.
Hi,
This is expected.
Thor uses the SBSA driver, so please use nvidia-smi to get the GPU info.
tegrastats is a Jetson customized tool that can get the utilization from nvgpu.
Orin and other previous Jetson devices use the nvgpu driver.
Thanks.
but nvidia-smi have many N/A info, is it possible to fix this N/A info?
±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.00 Driver Version: 580.00 CUDA Version: 13.0 |
±----------------------------------------±-----------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA Thor Off | 00000000:01:00.0 Off | N/A |
| N/A N/A N/A N/A / N/A | Not Supported | 4% Default |
| | | Disabled |
±----------------------------------------±-----------------------±---------------------+
±----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 15834 G /usr/lib/xorg/Xorg 0MiB |
| 0 N/A N/A 16028 G /usr/bin/gnome-shell 0MiB |
| 0 N/A N/A 16521 G /usr/libexec/gnome-initial-setup 0MiB |
| 0 N/A N/A 28305 C ./client 0MiB |
±----------------------------------------------------------------------------------------+
Hi,
Currently, the support for Thor is still limited but we keep extending the support.
Which function do you need? We can check it for you.
Thanks.
thanks for your help.
is it possible that nvidia-smi can get the gpu memory usage like NVIDIA L4?
how can we get the vic utilization?
±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.133.20 Driver Version: 570.133.20 CUDA Version: 12.8 |
|-----------------------------------------±-----------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA L4 Off | 00000000:01:00.0 Off | 0 |
| N/A 51C P8 13W / 72W | 23MiB / 23034MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
±----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 1251 G /usr/lib/xorg/Xorg 4MiB |
| 0 N/A N/A 10238 G /usr/lib/xorg/Xorg 4MiB |
±----------------------------------------------------------------------------------------+
Hi,
We will check with the internal team about the memory status.
But VIC is a Jetson-specific hardware so you will need to use tegrastats instead.
Thanks.
the problem is tegrastats can’t show vic info. even we use vic, still show 0% vic.
hi,
let’s have a summary.
- memory info of every gpu process
- vic’s utlilization, orin can get vic utlilization from jtop, but thor can’t.
please help to check whether we can get these two info.
Hi,
nvidia-smi cannot show memory information since igpu doesn’t have memory inside.
So please use tegrastats instead.
We can see VIC utilization with tegrastats.
Could you give it a try?
11-20-2025 11:17:08 RAM 9361/125772MB (lfb 9x4MB) ... VIC 14%@54 ...
Thanks.
Hi, Aasta
i checked it again, we can get vic utilization with tegrastats.
still have two problem.
- memory info
tegrastats only get the total memory info, it can’t get every gpu processs’s memory. but Minto give me a new driver from jetpack7.1, nvidia-smi can get memory info. maybe memory info will be fixed at next jetpack version.
2 . sm utilization info
at rtx4000, we use nvidia-smi pmon to get every process’s sm utilization, but nvidia-smi pmon can’t work at thor. this is a problem, please help check whether can support nvidia-smi pmon function?
Hi,
but Minto give me a new driver from jetpack7.1, nvidia-smi can get memory info.
Could you share more information about this (like a log) so we can check it with the internal team?
We will check the pmon part with our internal team and update later.
Thanks.
Hi,
Thanks for the feedback.
We will check with our internal team and update.
Thanks.
Hi,
nvidia-smi from JetPack 7.1 will support per-process system memory utilization.
But total memory usage is still unsupported.
nvidia-smi pmon is currently not supported on Thor.
Thanks.
Hi,
if don’t have total memory usage, it is ok for us.
but if nvidia-smi pmon is not supported, we can’t get every process’s gpu utilization, it is hard for us to debug performance issue.
is there any schedule to support nvidia-smi pmon? or is there any other method to get every process’s gpu utilization?
nvidia-smi dmon only can get the total gpu utilization, but can’t get every process’s gpu utilization.
is it possible to use nvml api to capture every process’s gpu utilization?
The might be help. pip install nvitop
https://github.com/XuehaiPan/nvitop
thanks for your reply. nvitop is the same with nvidia-smi. can’t show every process’s gpu utilization.
Don’t know if this is granular enough, or absolutely correct values since Thor uses shared memory, but most recent version of jtop shows the total amount of memory for a process on 2GPU page.
Here’s jtop installation instructions.
yes, jtop can show every process’s memory info, the same as jetpack7.1’s nvidia-smi.
the current problem is we can’t get every process’s gpu utilization info.