Vulkan crash

Robotics & Edge Computing Jetson & Embedded Systems Jetson Thor
,
1

I am using Qt 6 over Vulkan and have what seems to be a driver crash.

The same application, with the same Qt version works perfectly on a Windows or Linux x86 as well as on a AGX Orin Dev Kit, so I guess the problem came from the Vulkan Thor driver.

What I can observe is in some complex conditions, a call to vkQueueSubmit returns a device lost error and I get some dmesg errors:

[ 3265.471356] tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffffa00: EMEM address decode error (EMEM decode error)
[ 3265.471391] tegra-mc 8108020000.memory-controller: ptcr: @0x0000000000000000: Read response with poison bit error status:0
[ 3265.481159] tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffffe00: EMEM address decode error (EMEM decode error)
[ 3265.493729] tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffff600: EMEM address decode error (EMEM decode error)
[ 3265.506301] tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffffb00: EMEM address decode error (EMEM decode error)
[ 3265.518874] tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffffc00: EMEM address decode error (EMEM decode error)
[ 3265.531480] tegra-mc 8108020000.memory-controller: ptcr: @0x0000000000000000: Read response with poison bit error status:0
[ 3265.542672] arm-smmu-v3 8806000000.iommu: EVTQ overflow detected -- events lost

I spent some time to narrow down the smaller sample I can have that still produce the crash.

I made this example available here with the steps to reproduce this crash:

3

Hi,
Please share the steps to replicate the issue. We will set up AGX Orin developer kit and AGX Thor developer kit to check the issue.

4

The steps to replicate the issue are detailed in the test repository README.

It sums up to install Qt, start the sample from the git repository and click the button.

5

Hi,
Please share the commands for the installation:

Qt 6.9.2, installed from:
https://download.qt.io/official_releases/online_installers/qt-online-installer-linux-arm64-online.run
Installed with minimal options

We are not familiar with QT and would need your help to provide more information.

6

It is a straigthforward visual installer, just run it with :

$ chmod +x qt-online-installer-linux-arm64-online.run
$ ./qt-online-installer-linux-arm64-online.run

Then for each page:

  • Welcome → Click next
  • Login → Create a free account by clicking on “Sign up“ an use it to log in
  • Open Source Obligations → Accept the terms
  • Installation options → Keep defaults or customize to keep only the Qt/Qt 6.9.2/Desktop entry
  • License Agreement → Accept the terms
  • Installation Summary → Next
  • Installing → Click Install
  • Finished → Click Finish

I know it is an extra step from your domain but I do not use Vulkan directly and it should trigger the crash on your driver side.

8

Hi,
We have not replicated the issue yet. Somehow it frequently stops while downloading QT package. Is there other way to run Vulkan application directly? Not sure if there is a way to reproduce the issue without QT installation.

9

Hi,

I have updated the test repository with the Qt binaries for Thor, built from their public sources:

https://download.qt.io/official_releases/qt/6.8/6.8.3/single/qt-everywhere-src-6.8.3.tar.xz

You just have to untar the Qt.tar.gz from the repository and launch the samples with it, as described in the updated README file.

Thank you

1 Like
10

Hi,
Thanks. If I do the steps on AGX Orin developer kit, it’s supposed to work, right? We would like to compare the result on Orin and Thor.

11

The previous Qt build was made for the Thor only and until the last Jetpack is available for Orin it wont work easily.

I spent some time to test and push the Orin Qt version on the repository.

I confirm that I still do NOT have this crash on the Orin.

You should be able to test and compare on both platforms now.

1 Like
12

Running github/Atlas42/thor-qt-vk-crash on Thor

./Qt/bin/qml -style Universal -r vulkan Main.qml
Device loss detected in vkQueueSubmit()
Graphics device lost, cleaning up scenegraph and releasing RHI

Oct 14 17:47:50 chithor kernel: NVRM: GPU at PCI:0000:01:00: GPU-a7c66ad2-6dbb-0ab8-c1a2-37ba6dba3600
Oct 14 17:47:50 chithor kernel: NVRM: Xid (PCI:0000:01:00): 69, pid=73862, name=QSGRenderThread, Class Error: channel 0x0000000d, Class 0000cd97, Offset 000017d8, Data 80000453, ErrorCode 0000000c
Oct 14 17:47:56 chithor /usr/libexec/gdm-x-session[2708]: (WW) NVIDIA: Wait for channel idle timed out.

Downloaded/installed opensource Qt/6.9.3/gcc_arm64

~/Qt/6.9.3/gcc_arm64/bin/qml -style Universal -r vulkan Main.qml
[Tue Oct 14 18:45:29 2025] NVRM: Xid (PCI:0000:01:00): 69, pid=99262, name=qml, Class Error: channel 0x0000000c, Class 0000cd97, Offset 000017d8, Data 80000453, ErrorCode 0000000c

journalctl -xe 
Oct 14 18:45:28 chithor kernel: NVRM: Xid (PCI:0000:01:00): 69, pid=99262, name=qml, Class Error: channel 0x0000000c, Class 0000cd97, Offset 000017d8, Data 80000453, ErrorCode 0000000c
Oct 14 18:45:34 chithor /usr/libexec/gdm-x-session[2708]: (WW) NVIDIA: Wait for channel idle timed out.

Ran some traces on Thor with Qt /6.9.3/gcc_arm64 opensource and attach the results.

Will do same on my Orin if any one wants.

strace -f -ttT -s 256 -yy -o qml.strace \
  ~/Qt/6.9.3/gcc_arm64/bin/qml -style Universal -r vulkan Main.qml

ltrace -f -tt -o qml.ltrace \
  ~/Qt/6.9.3/gcc_arm64/bin/qml -style Universal -r vulkan Main.qml

LD_DEBUG=libs,bindings ~/Qt/6.9.3/gcc_arm64/bin/qml -style Universal -r vulkan Main.qml 2> qml.lddebug.txt

QT_DEBUG_PLUGINS=1 \
QT_LOGGING_RULES="*.debug=true;qt.qml.binding=false" \
~/Qt/6.9.3/gcc_arm64/bin/qml -style Universal -r vulkan Main.qml 2> qt.plugins.txt

VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
VK_LOADER_DEBUG=all \
~/Qt/6.9.3/gcc_arm64/bin/qml -style Universal -r vulkan Main.qml 2> vulkan.validation.txt

vulkan.validation.txt (29.8 KB)

qml.strace.txt (5.2 MB)

qml.ltrace.txt (43.4 KB)

ltrace.txt (201 Bytes)

qml.lddebug.txt (16.9 MB)

qt.plugins.txt (802.5 KB)

1 Like
14

Tested “~/thor-qt-vk-crash/Qt/bin/qml -style Universal -r vulkan Main.qml“ On Orin. No problems

Don’t know these matter, but it is a difference.

On Orin there are zero auto installed qt5, qt6 or qml packages.
:~$ apt search qt|grep installed; apt search qml|grep installed

On Thor there are 39 Qt{5,6} and qml packages installed. See attached

Qt_apt_packages_installed_on_Thor.txt (3.0 KB)

15

Thank you for your help.

The first thing I did when I installed the Thor was to remove the provided Qt installs, to avoid conflicts.

So I can confirm it crashes without those packages

16

Following done on Thor. These two work. but cause display flicker.

QT_XCB_GL_INTEGRATION=xcb_egl qml -style Universal Main.qml

QSG_RHI_BACKEND=opengl qml -style Universal Main.qml

  1. QSG_RENDER_LOOP=basic QSG_RHI_PREFER_COMPAT=1 QSG_RHI_BACKEND=vulkan
    qml -style Universal -r vulkan Main.qml

Device loss detected in vkQueueSubmit()
Graphics device lost, cleaning up scenegraph and releasing RHIs

  1. QSG_RENDER_LOOP=basic QSG_RHI_BACKEND=vulkan QSG_RHI_SWAPCHAIN_MAX_BUFFER_COUNT=2
    qml -style Universal -r vulkan Main.qml

No error and qt app worked and closed when x clicked. app execution caused display to flicker and flash.

  1. QSG_RENDER_LOOP=basic QSG_RHI_BACKEND=vulkan QSG_RHI_DISABLE_VK_EXTENSIONS=VK_KHR_fragment_shading_rate
    qml -style Universal -r vulkan Main.qml

Device loss detected in vkQueueSubmit()
Graphics device lost, cleaning up scenegraph and releasing RHIs

  1. QT_QPA_PLATFORM=wayland QSG_RENDER_LOOP=basic QSG_RHI_BACKEND=vulkan
    qml -style Universal -r vulkan Main.qml

Failed to create wl_display (No such file or directory)
qt.qpa.plugin: Could not load the Qt platform plugin “wayland” in “” even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: xcb, wayland-egl, wayland, eglfs, minimal, offscreen, minimalegl, linuxfb, vkkhrdisplay, vnc.
Aborted (core dumped)

  1. QT_QPA_PLATFORM=xcb QSG_RENDER_LOOP=basic QSG_RHI_BACKEND=vulkan
    qml -style Universal -r vulkan Main.qml

No error and qt app worked and closed when x clicked. app execution caused display to flicker and flash.

  1. sudo nvpmodel -m 0 ; sudo jetson_clocks
    QSG_RENDER_LOOP=basic QSG_RHI_BACKEND=vulkan qml -style Universal -r vulkan Main.qml

Device loss detected in vkQueueSubmit()
Graphics device lost, cleaning up scenegraph and releasing RHIs
And, most pronounced display flicker/flash

  1. VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_LOADER_DEBUG=all QSG_INFO=1 QSG_RHI_DEBUG_LAYER=1
    qml -style Universal -r vulkan Main.qml 2> vk_debug.log.txt

Attached vk_debug.log.txt (33.6 KB)

journalctl -k --since “5 minutes ago” | tail -n 100

Oct 16 16:56:52 chithor kernel: NVRM: Xid (PCI:0000:01:00): 69, pid=21328, name=qml, Class Error: channel 0x0000000c, Class 0000cd97, Offset 000017d8, Data 80000453, ErrorCode 0000000c
Oct 16 16:57:09 chithor kernel: tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffff880: EMEM address decode error (EMEM decode error)
Oct 16 16:57:09 chithor kernel: tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffffa00: EMEM address decode error (EMEM decode error)
Oct 16 16:57:09 chithor kernel: tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffffc00: EMEM address decode error (EMEM decode error)
Oct 16 16:57:09 chithor kernel: tegra-mc 8108020000.memory-controller: ptcr: @0x0000000000000000: Read response with poison bit error status:0
Oct 16 16:57:09 chithor kernel: tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffffe00: EMEM address decode error (EMEM decode error)
Oct 16 16:57:09 chithor kernel: tegra-mc 8108020000.memory-controller: dispr: non-secure read @0x0000fffffffff980: EMEM address decode error (EMEM decode error)
Oct 16 16:57:09 chithor kernel: arm-smmu-v3 8806000000.iommu: EVTQ overflow detected -- events lost
Oct 16 16:57:09 chithor kernel: arm_smmu_evtq_thread: 23086 callbacks suppressed
Oct 16 16:57:09 chithor kernel: arm-smmu-v3 8806000000.iommu: event 0x10 received:
Oct 16 16:57:09 chithor kernel: arm-smmu-v3 8806000000.iommu:         0x0000090000000010
Oct 16 16:57:09 chithor kernel: arm-smmu-v3 8806000000.iommu:         0x0000020800000000
Oct 16 16:57:09 chithor kernel: arm-smmu-v3 8806000000.iommu:         0x0000007ffae5e880
Oct 16 16:57:09 chithor kernel: arm-smmu-v3 8806000000.iommu:         0x0000000000000000
17

Hi,
We are debugging it and it shall be fixed in next Jetpack 7 release.

1 Like
18

Great !

Thank you for your help

19

I have tried using QSG_RENDER_LOOP=basic and QSG_RHI_SWAPCHAIN_MAX_BUFFER_COUNT=2 options.

When using the basic render loop, the test project stop crashing, but when testing on my whole application, I still have the same crashes, so it just hides the problem.

I will wait for the next release.