We are using the Orin NX Devkit,
environment information is as follows
nvidia@ubuntu:~$ cat /etc/nv_tegra_release
# R36 (release), REVISION: 3.0, GCID: 36191598, BOARD: generic, EABI: aarch64, DATE: Mon May 6 17:34:21 UTC 2024
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia
nvidia@ubuntu:~$ nvidia-smi
Wed Mar 26 14:23:09 2025
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 540.3.0 Driver Version: N/A CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Orin (nvgpu) N/A | N/A N/A | N/A |
| N/A N/A N/A N/A / N/A | Not Supported | N/A N/A |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
We were running bitbucket pipeline in this environment and have been experiencing errors running the runner lately.
On a related note, there was a network related issue with the latest docker engine on the nvidia forum,
The following logs are from a later version that says the issue is fixed.
I am getting an error in nvidia jetpack environment, Orin NX Devkit when i try to do docker in docker,
I was hoping someone could give me a hint on how to fix it.
nvidia@ubuntu:~$ docker run --privileged --name dind -d docker
4b722935015bdd034d3ed7b0c60fd3b948513ac64370a9cf2d53076003a6f370
nvidia@ubuntu:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4b722935015b docker "dockerd-entrypoint.…" 2 seconds ago Up 1 second 2375-2376/tcp dind
nvidia@ubuntu:~$ docker logs -f dind
Certificate request self-signature ok
subject=CN=docker:dind server
/certs/server/cert.pem: OK
Certificate request self-signature ok
subject=CN=docker:dind client
/certs/client/cert.pem: OK
cat: can't open '/proc/net/arp_tables_names': No such file or directory
iptables v1.8.11 (nf_tables)
time="2025-03-26T05:18:28.844168004Z" level=info msg="Starting up"
time="2025-03-26T05:18:28.845285865Z" level=info msg="containerd not running, starting managed containerd"
time="2025-03-26T05:18:28.847866571Z" level=info msg="started new containerd process" address=/var/run/docker/containerd/containerd.sock module=libcontainerd pid=73
time="2025-03-26T05:18:28.862768261Z" level=info msg="starting containerd" revision=05044ec0a9a75232cad458027ca83437aae3f4da version=v1.7.27
time="2025-03-26T05:18:28.880147236Z" level=info msg="loading plugin \"io.containerd.event.v1.exchange\"..." type=io.containerd.event.v1
time="2025-03-26T05:18:28.880214946Z" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1
time="2025-03-26T05:18:28.880642200Z" level=info msg="loading plugin \"io.containerd.warning.v1.deprecations\"..." type=io.containerd.warning.v1
time="2025-03-26T05:18:28.880667256Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.blockfile\"..." type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.880770581Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.blockfile\"..." error="no scratch file generator: skip plugin" type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.880789141Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.880807220Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." error="devmapper not configured: skip plugin" type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.880818996Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.880906994Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.881206187Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.888216034Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"ip: can't find device 'aufs'\\nmodprobe: can't change directory to '/lib/modules': No such file or directory\\n\"): skip plugin" type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.888248482Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.888396190Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2025-03-26T05:18:28.888413502Z" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
time="2025-03-26T05:18:28.888431453Z" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1
time="2025-03-26T05:18:28.888518587Z" level=info msg="metadata content store policy set" policy=shared
time="2025-03-26T05:18:28.891380279Z" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1
time="2025-03-26T05:18:28.891435413Z" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1
time="2025-03-26T05:18:28.891458261Z" level=info msg="loading plugin \"io.containerd.lease.v1.manager\"..." type=io.containerd.lease.v1
time="2025-03-26T05:18:28.891478804Z" level=info msg="loading plugin \"io.containerd.streaming.v1.manager\"..." type=io.containerd.streaming.v1
time="2025-03-26T05:18:28.891496980Z" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1
time="2025-03-26T05:18:28.891703663Z" level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..." type=io.containerd.monitor.v1
time="2025-03-26T05:18:28.891963785Z" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2
time="2025-03-26T05:18:28.892132516Z" level=info msg="loading plugin \"io.containerd.runtime.v2.shim\"..." type=io.containerd.runtime.v2
time="2025-03-26T05:18:28.892160964Z" level=info msg="loading plugin \"io.containerd.sandbox.store.v1.local\"..." type=io.containerd.sandbox.store.v1
time="2025-03-26T05:18:28.892188803Z" level=info msg="loading plugin \"io.containerd.sandbox.controller.v1.local\"..." type=io.containerd.sandbox.controller.v1
time="2025-03-26T05:18:28.892206691Z" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1
time="2025-03-26T05:18:28.892222402Z" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1
time="2025-03-26T05:18:28.892237058Z" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1
time="2025-03-26T05:18:28.892254658Z" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1
time="2025-03-26T05:18:28.892273249Z" level=info msg="loading plugin \"io.containerd.service.v1.introspection-service\"..." type=io.containerd.service.v1
time="2025-03-26T05:18:28.892287809Z" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1
time="2025-03-26T05:18:28.892302304Z" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
time="2025-03-26T05:18:28.892316800Z" level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..." type=io.containerd.service.v1
time="2025-03-26T05:18:28.892347327Z" level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892365343Z" level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892380927Z" level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892394910Z" level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892410462Z" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892425501Z" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892439517Z" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892454653Z" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892469628Z" level=info msg="loading plugin \"io.containerd.grpc.v1.sandbox-controllers\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892490716Z" level=info msg="loading plugin \"io.containerd.grpc.v1.sandboxes\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892505436Z" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892520155Z" level=info msg="loading plugin \"io.containerd.grpc.v1.streaming\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892534907Z" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892552378Z" level=info msg="loading plugin \"io.containerd.transfer.v1.local\"..." type=io.containerd.transfer.v1
time="2025-03-26T05:18:28.892583738Z" level=info msg="loading plugin \"io.containerd.grpc.v1.transfer\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892601497Z" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892619993Z" level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..." type=io.containerd.internal.v1
time="2025-03-26T05:18:28.892682455Z" level=info msg="loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." type=io.containerd.tracing.processor.v1
time="2025-03-26T05:18:28.892703479Z" level=info msg="skip loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." error="skip plugin: tracing endpoint not configured" type=io.containerd.tracing.processor.v1
time="2025-03-26T05:18:28.892715350Z" level=info msg="loading plugin \"io.containerd.internal.v1.tracing\"..." type=io.containerd.internal.v1
time="2025-03-26T05:18:28.892728918Z" level=info msg="skip loading plugin \"io.containerd.internal.v1.tracing\"..." error="skip plugin: tracing endpoint not configured" type=io.containerd.internal.v1
time="2025-03-26T05:18:28.892739958Z" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1
time="2025-03-26T05:18:28.892755957Z" level=info msg="loading plugin \"io.containerd.nri.v1.nri\"..." type=io.containerd.nri.v1
time="2025-03-26T05:18:28.892769493Z" level=info msg="NRI interface is disabled by configuration."
time="2025-03-26T05:18:28.893103757Z" level=info msg=serving... address=/var/run/docker/containerd/containerd-debug.sock
time="2025-03-26T05:18:28.893207307Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
time="2025-03-26T05:18:28.893297992Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock
time="2025-03-26T05:18:28.893327272Z" level=info msg="containerd successfully booted in 0.031588s"
time="2025-03-26T05:18:28.901607617Z" level=info msg="OTEL tracing is not configured, using no-op tracer provider"
time="2025-03-26T05:18:28.911965896Z" level=info msg="Creating a containerd client" address=/var/run/docker/containerd/containerd.sock timeout=1m0s
time="2025-03-26T05:18:28.925590529Z" level=info msg="Loading containers: start."
time="2025-03-26T05:18:28.928485820Z" level=warning msg="Could not load necessary modules for IPSEC rules: protocol not supported"
time="2025-03-26T05:18:29.307269529Z" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
time="2025-03-26T05:18:29.307507379Z" level=info msg="stopping healthcheck following graceful shutdown" module=libcontainerd
time="2025-03-26T05:18:29.307558194Z" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby
failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to register "bridge" driver: failed to add jump rules to ipv4 NAT table: failed to append jump rules to nat-PREROUTING: (iptables failed: iptables --wait -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER: Warning: Extension addrtype revision 0 not supported, missing kernel module?
iptables v1.8.11 (nf_tables): CHAIN_ADD failed (No such file or directory): chain PREROUTING
(exit status 4))
nvidia@ubuntu:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
nvidia@ubuntu:~$ docker version
Client: Docker Engine - Community
Version: 28.0.4
API version: 1.48
Go version: go1.23.7
Git commit: b8034c0
Built: Tue Mar 25 15:07:33 2025
OS/Arch: linux/arm64
Context: default
Server: Docker Engine - Community
Engine:
Version: 28.0.4
API version: 1.48 (minimum version 1.24)
Go version: go1.23.7
Git commit: 6430e49
Built: Tue Mar 25 15:07:33 2025
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.7.26
GitCommit: 753481ec61c7c8955a23d6ff7bc8e4daed455734
nvidia:
Version: 1.2.5
GitCommit: v1.2.5-0-g59923ef
docker-init:
Version: 0.19.0
GitCommit: de40ad0