Text-generation-webui cannot work

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
1

I follow the text-generation-webui - NVIDIA Jetson AI Lab to try text-generation-webui from scratch.

git clone GitHub - dusty-nv/jetson-containers: Machine Learning Containers for NVIDIA Jetson and JetPack-L4T
bash jetson-containers/install.sh

Pop out indicated it is ok.

Then

jetson-containers run $(autotag text-generation-webui)

Pop out below error:
orin@orin-desktop:~/jetson-containers$ jetson-containers run $(autotag text-generation-webui)
Namespace(packages=[‘text-generation-webui’], prefer=[‘local’, ‘registry’, ‘build’], disable=[‘’], user=‘dustynv’, output=‘/tmp/autotag’, quiet=False, verbose=False)
– L4T_VERSION=36.4.4 JETPACK_VERSION=6.2.1 CUDA_VERSION=12.6
– Finding compatible container image for [‘text-generation-webui’]
[sudo] password for orin:
text-generation-webui:r36.4.tegra-aarch64-cu126-22.04-cuda_12.6
V4L2_DEVICES:

DISPLAY environmental variable is already set: “:0”

localuser:root being added to access control list

ARM64 architecture detected

Jetson Detected

SYSTEM_ARCH=tegra-aarch64

  • sudo docker run --runtime nvidia --env NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics -it --rm --network host --shm-size=8g --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.sock --volume /home/orin/jetson-containers/data:/data -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --device /dev/snd -e PULSE_SERVER=unix:/run/user/1000/pulse/native -v /run/user/1000/pulse:/run/user/1000/pulse --device /dev/bus/usb -e DISPLAY=:0 -v /tmp/.X11-unix/:/tmp/.X11-unix -v /tmp/.docker.xauth:/tmp/.docker.xauth -e XAUTHORITY=/tmp/.docker.xauth --device /dev/i2c-0 --device /dev/i2c-1 --device /dev/i2c-2 --device /dev/i2c-4 --device /dev/i2c-5 --device /dev/i2c-7 --device /dev/i2c-9 --name jetson_container_20250729_081308 text-generation-webui:r36.4.tegra-aarch64-cu126-22.04-cuda_12.6
    docker: Error response from daemon: unknown or invalid runtime name: nvidia

Run ‘docker run --help’ for more information

How should I fix this error? Thanks.
I referred to Running ai docker containers on jetson orin nano with gpu support - #9 by AastaLLL link, the error is same.

4

Hi,

docker: Error response from daemon: unknown or invalid runtime name: nvidia

Please help us verify the NVIDIA container toolkit.

https://www.jetson-ai-lab.com/tips_ssd-docker.html#docker

Add default runtime in /etc/docker/daemon.json

Thanks.

5

I use your link to install container toolkit. Then I want to use container to install Ollama, find below error:

rin@orin-desktop:~$ jetson-containers run --name ollama $(autotag ollama)
Namespace(packages=[‘ollama’], prefer=[‘local’, ‘registry’, ‘build’], disable=[‘’], user=‘dustynv’, output=‘/tmp/autotag’, quiet=False, verbose=False)
– L4T_VERSION=36.4.4 JETPACK_VERSION=6.2.1 CUDA_VERSION=12.6
– Finding compatible container image for [‘ollama’]

Found compatible container dustynv/ollama:0.6.8-r36.4-cu126-22.04 (2025-05-06, 5.1GB) - would you like to pull it? [Y/n] Y
dustynv/ollama:0.6.8-r36.4-cu126-22.04
V4L2_DEVICES:

DISPLAY environmental variable is already set: “:0”

localuser:root being added to access control list

ARM64 architecture detected

Jetson Detected

SYSTEM_ARCH=tegra-aarch64

  • docker run --runtime nvidia --env NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics -it --rm --network host --shm-size=8g --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.sock --volume /home/orin/jetson-containers/data:/data -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --device /dev/snd -e PULSE_SERVER=unix:/run/user/1000/pulse/native -v /run/user/1000/pulse:/run/user/1000/pulse --device /dev/bus/usb -e DISPLAY=:0 -v /tmp/.X11-unix/:/tmp/.X11-unix -v /tmp/.docker.xauth:/tmp/.docker.xauth -e XAUTHORITY=/tmp/.docker.xauth --device /dev/i2c-0 --device /dev/i2c-1 --device /dev/i2c-2 --device /dev/i2c-4 --device /dev/i2c-5 --device /dev/i2c-7 --device /dev/i2c-9 --name ollama dustynv/ollama:0.6.8-r36.4-cu126-22.04
    docker: unknown server OS:

Run ‘docker run --help’ for more information

How should I fix?

6

Hi,

Could you run the below command locally to see if it can work?

$ docker run --runtime nvidia --env NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics -it --rm --network host --shm-size=8g --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.sock --volume /home/orin/jetson-containers/data:/data -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --device /dev/snd -e PULSE_SERVER=unix:/run/user/1000/pulse/native -v /run/user/1000/pulse:/run/user/1000/pulse --device /dev/bus/usb -e DISPLAY=:0 -v /tmp/.X11-unix/:/tmp/.X11-unix -v /tmp/.docker.xauth:/tmp/.docker.xauth -e XAUTHORITY=/tmp/.docker.xauth --device /dev/i2c-0 --device /dev/i2c-1 --device /dev/i2c-2 --device /dev/i2c-4 --device /dev/i2c-5 --device /dev/i2c-7 --device /dev/i2c-9 --name ollama dustynv/ollama:0.6.8-r36.4-cu126-22.04

Thanks.

7

You did not reply my question. I want to know how to debug Jetson apps based on the pop out, NOT just run an app.

8

Hi,

The error occurs when running the docker run command above.
Could you try to run it directly (without using the jetson-containers wrapper) to see if it can work?

docker: unknown server OS:

The error you met above is more like an environment setting problem.
Which JetPack version do you use?

Thanks.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.