Nvargus-daemon problem integrating camera

Robotics & Edge Computing Jetson & Embedded Systems Jetson Thor
, , ,
1

I’m bringing up a custom carrier board with a custom camera and facing some issues with nvargus-daemon.

I have integrated the camera. Have a device node in /dev/video0 and is able to get picture from it with v4l. But when trying to use `nvarguscamerasrc` I can’t getting it to work.

# gst-launch-1.0 nvarguscamerasrc sensor-id=0 num-buffers=1 !  'video/x-raw(memory:NVMM), width=2472, height=2064, framerate=60/1' !  nvvidconv !  nvjpegenc ! filesink location=capture.jpg
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Error generated. gstnvarguscamerasrc.cpp, execute:803 No cameras available
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:00.003857129
Setting pipeline to NULL ...
Freeing pipeline ...

And here is the log from nvargus-daemon that I think is where the error is:

Jan 01 01:31:12 hostname nvargus-daemon[5185]: === gst-launch-1.05549)[5549]: Connection established (FFFF8F7398E0)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: libEGL warning: pci id for fd 9: 10de:2b00, driver (null)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: pci id for fd 10: 10de:2b00, driver (null)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: kmsro: driver missing
Jan 01 01:31:12 hostname nvargus-daemon[5185]: libEGL warning: egl: failed to create dri2 screen
Jan 01 01:31:12 hostname nvargus-daemon[5185]: libEGL warning: pci id for fd 9: 10de:2b00, driver (null)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: pci id for fd 10: 10de:2b00, driver (null)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: kmsro: driver missing
Jan 01 01:31:12 hostname nvargus-daemon[5185]: libEGL warning: egl: failed to create dri2 screen
Jan 01 01:31:12 hostname nvargus-daemon[5185]: libEGL warning: pci id for fd 9: 10de:2b00, driver (null)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: SCF: Error 0x00000006: Unable to initialize EGL (in src/services/gl/GLService.cpp, function initialize(), line 144)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: SCF: Error 0x00000006:  (propagating from src/services/gl/GLService.cpp, function startService(), line 46)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: SCF: Error 0x00000006:  (propagating from src/components/ServiceHost.cpp, function startServices(), line 142)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: CaptureService:stop: is requested --------------------------
Jan 01 01:31:12 hostname nvargus-daemon[5185]: SCF: Error 0x00000006:  (propagating from src/api/CameraDriver.cpp, function initialize(), line 182)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: SCF: Error 0x00000008: Services are already stopped (in src/components/ServiceHost.cpp, function stopServicesInternal(), line 193)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: SCF: Error 0x00000006:  (propagating from src/api/CameraDriver.cpp, function getCameraDriver(), line 120)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: (Argus) Error 0x00000006:  (propagating from src/api/GlobalProcessState.cpp, function createCameraProvider(), line 210)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: === gst-launch-1.05549)[5549]: CameraProvider failed to initialize
Jan 01 01:31:12 hostname nvargus-daemon[5185]: === gst-launch-1.05549)[5549]: Connection closed (FFFF8F7398E0)
Jan 01 01:31:12 hostname nvargus-daemon[5185]: === gst-launch-1.05549)[5549]: Connection cleaned up (FFFF8F7398E0)

It would be great to get some insights here.

3

I’m facing same issues here :(

4

The error tell incorrect context in tegra_camera_platorm{} make argus can’t detect any camera.

Error generated. gstnvarguscamerasrc.cpp, execute:803 No cameras available
5

@ShaneCCC and can you give us a hint why it can not detect any camera and where to start digging? I haven’t found any open source code to argus-daemon so it is really hard to debug.

I have my cameras available in the system. It is possible to use v4l-ctl to capture a picture. In what way do argus-daemon look for the cameras?

6
  1. Dump the device tree to confirm the tegra_camera_platform{}
sudo dtc -I fs -O dts -o extracted_proc.dts /sys/firmware/devicetree/base

Get the device info by below command.

v4l2-ctl --list-devices
v4l2-ctl --all
7

@ShaneCCC here is the output from my system:

# v4l2-ctl --list-devices
NVIDIA Tegra Video Input Device (platform:tegra-capture-vi):
        /dev/media0

vi-output, imx568 18-003d (platform:tegra-capture-vi:2):
        /dev/video0

vi-output, imx568 19-003c (platform:tegra-capture-vi:4):
        /dev/video1


# v4l2-ctl --all
Driver Info:
        Driver name      : tegra-video
        Card type        : vi-output, imx568 18-003d
        Bus info         : platform:tegra-capture-vi:2
        Driver version   : 6.8.12
        Capabilities     : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Priority: 2
Video input : 0 (Camera 2: ok)
Format Video Capture:
        Width/Height      : 2472/2064
        Pixel Format      : 'RG10' (10-bit Bayer RGRG/GBGB)
        Field             : None
        Bytes per Line    : 4944
        Size Image        : 10204416
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             : 
Streaming Parameters Video Capture:
        Capabilities     : timeperframe
        Frames per second: 79.000 (79/1)
        Read buffers     : 0

Camera Controls

                     group_hold 0x009a2003 (bool)   : default=0 value=0 flags=execute-on-write
                    sensor_mode 0x009a2008 (int64)  : min=0 max=3 step=1 default=0 value=0 flags=slider
                           gain 0x009a2009 (int64)  : min=10 max=241 step=1 default=10 value=10 flags=slider
                       exposure 0x009a200a (int64)  : min=7 max=12651 step=1 default=10000 value=7 flags=slider
                     frame_rate 0x009a200b (int64)  : min=79000000 max=79000000 step=1 default=79000000 value=79000000 flags=slider
           sensor_configuration 0x009a2032 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[22] flags=read-only, volatile, has-payload
         sensor_mode_i2c_packet 0x009a2033 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[1026] flags=read-only, volatile, has-payload
      sensor_control_i2c_packet 0x009a2034 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[1026] flags=read-only, volatile, has-payload
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
                                0: 0 (0x0)
                                1: 1 (0x1)
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
                                0: 0 (0x0)
                                1: 1 (0x1)
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0 (1 0x1)
                                0: 1 (0x1)
                                1: 65536 (0x10000)
                                2: 131072 (0x20000)
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][18] flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][36] flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
    override_capture_timeout_ms 0x009a206f (int)    : min=-1 max=2147483647 step=1 default=2500 value=2500
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=3 flags=read-only

extracted_proc.dts.txt (338.7 KB)

8

Device tree looks fine.

Could be EGL driver didn’t load.

Have below command first.

export DISPLAY=:0
gst-launch-1.0 .....

Jan 01 01:31:12 hostname nvargus-daemon[5185]: SCF: Error 0x00000006: Unable to initialize EGL (in src/services/gl/GLService.cpp, function initialize(), line 144)
9

I realized I had several EGL libraries in my rootfs and the one that was loaded was `libEGL_mesa.so.0`. This didn’t have support for Thor and the `Unable to initialize EGL` error was probably due to that.

I have changed my rootfs and now the library `libEGL_nvidia.so.0` is loaded by `nvargus-daemon`. The result at the top is similar, no cameras available for argus but under the hood the error message has changed a bit to `No EGL device available`.

Here is the new output:

# DISPLAY=:0 /usr/sbin/nvargus-daemon                   
=== NVIDIA Libargus Camera Service (0.99.33)
=== Listening for connections...
=== gst-launch-1.05221)[5221]: Connection established (FFFF75E998E0)
��INFO: END TASK:MB��
INFO: enter idle task.
INFO: END TASK:MB��
��ph =��ter idle ta��GPC ��sk.
        ��> logic map: 0=>0 1=>2 2=>1
��SCF: Error 0x00000002: No EGL device available (in src/services/gl/GLService.cpp, function initialize(), line 128)
SCF: Error 0x00000002:  (propagating from src/services/gl/GLService.cpp, function startService(), line 46)
SCF: Error 0x00000002:  (propagating from src/components/ServiceHost.cpp, function startServices(), line 142)
CaptureService:stop: is requested --------------------------
SCF: Error 0x00000008: Service not running (in src/services/gl/GLService.cpp, function stopService(), line 55)
SCF: Error 0x00000008:  (propagating from src/components/ServiceHost.cpp, function stopServicesInternal(), line 208)
SCF: Error 0x00000002:  (propagating from src/api/CameraDriver.cpp, function initialize(), line 182)
SCF: Error 0x00000008: Services are already stopped (in src/components/ServiceHost.cpp, function stopServicesInternal(), line 193)
SCF: Error 0x00000002:  (propagating from src/api/CameraDriver.cpp, function getCameraDriver(), line 120)
(Argus) Error 0x00000002:  (propagating from src/api/GlobalProcessState.cpp, function createCameraProvider(), line 210)
=== gst-launch-1.05221)[5221]: CameraProvider failed to initialize
=== gst-launch-1.05221)[5221]: Connection closed (FFFF75E998E0)
=== gst-launch-1.05221)[5221]: Connection cleaned up (FFFF75E998E0)

I’m building my rootfs with meta-tegra and Yocto. I have used LD_DEBUG and strace to see what happens in nvargus-daemon and I share it in the attached file. Can you please take a look and try to help out to fins the reason that no EGL devices is found?

nvargus.log (446.0 KB)

10

Could you flash the system by sdkmanager again.

11

No. It is a custom board with a Thor module.

12

I have a Thor dev-kit with JP7 and my custom board with custom image. My `nvargus_daemon`seems to fail when trying to initialize EGL and for that reason I have this EGL reproducer that I run on both systems:

#include <stdio.h>
#include <dlfcn.h>
#include <EGL/egl.h>

int main() {
    void *handle;
    handle = dlopen("libnvidia-eglcore.so.580.00", RTLD_NOW);
    if (!handle) {
        fprintf(stderr, "Failed to load libnvidia-eglcore.so.580.00: %s\n", dlerror());
        return 1;
    }
    printf("libnvidia-eglcore.so.580.00 loaded successfully.\n");

    // Try a simple EGL query
    EGLDisplay dpy = eglGetDisplay(EGL_DEFAULT_DISPLAY);
    if (dpy == EGL_NO_DISPLAY) {
        fprintf(stderr, "eglGetDisplay failed\n");
        dlclose(handle);
        return 1;
    }
    if (!eglInitialize(dpy, NULL, NULL)) {
        fprintf(stderr, "eglInitialize failed\n");
        dlclose(handle);
        return 1;
    }

    printf("EGL initialized successfully.\n");

    eglTerminate(dpy);
    dlclose(handle);
    return 0;
}

On my dev-kit the output is:

# ./test_egl
libnvidia-eglcore.so.580.00 loaded successfully.
EGL initialized successfully.

On my custom board the output is:

# ./test_egl 
libnvidia-eglcore.so.580.00 loaded successfully.
eglInitialize failed

So the function `eglInitialize` have different return values on the different platforms. I have made sure the setup of libraries are similar. Tried to work with `strace` and `LD_DEBUG` to understand the difference. I attach the logs from the two. Just before the return it starts working with `/dev/nvidia0` and sending ioctl’s to that node.

I have observed one difference in the reference of the kernel modules. On my custom board there are no users of the drivers `nvidia_drm` or `tegra_drm`. On the dev-kit during execution the `/dev/nvidia1` is opened but that does not happen on my custom board, but it is present in both. Also some difference when searhing for drm in the kernel log that I show here:

dev-kit:

# dmesg | grep -i drm
[    7.096887] systemd[1]: Starting [email protected] - Load Kernel Module drm...
[    7.762612] [drm] Initialized tegra 1.0.99 20120330 for 8181200000.host1x.drm on minor 0
[   11.995036]  tegra234_oc_event(O) r8126(O) rfkill snd_hda_codec_hdmi tegra23x_psc(O) snd_soc_rt5640 nvethernet(O) tegra_aconnect snd_soc_rl6231 thermal_trip_event(O) tegra_cactmon_mc_all(O) nvidia_vrs_pseq(O) lm90 snd_hda_tegra snd_hda_codec at24 mttcan(O) snd_hda_core host1x_fence(O) nvpps(O) can_dev pwm_tegra_tachometer(O) nvidia_cspmu ramoops spi_tegra114 tegra_dce(O) reed_solomon arm_cspmu_module tpm_ftpm_tee camera_diagnostics(O) nvhost_isp5(O) tegra_capture_isp(O) tegra_camera(O) v4l2_dv_timings host1x_nvhost(O) tegra_drm(O) tegra_wmark(O) nvhwpm(O) drm_display_helper drm_dp_aux_bus cec drm_kms_helper host1x(O) tegra_camera_platform(O) mc_utils(O) capture_ivc(O) v4l2_fwnode v4l2_async videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videodev videobuf2_common mc camchar(O) rtcpu_debug(O) tegra_camera_rtcpu(O) ivc_bus(O) hsp_mailbox_client(O) nvme_fabrics fuse drm nfnetlink ip_tables x_tables ipv6 pwm_fan pwm_tegra tegra_bpmp_thermal tegra_xudc uas ucsi_ccg typec_ucsi typec nvme nvme_core phy_tegra194_p2u
[   12.033654] [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
[   12.995781] [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 1
[   12.995794] nvidia 0000:01:00.0: [drm] No compatible format found
[   12.995809] nvidia 0000:01:00.0: [drm] Cannot find any crtc or sizes
[   12.995837] [drm] [nvidia-drm] [GPU ID 0x00020000] Loading driver
[   13.340936] [drm] Initialized nvidia-drm 0.0.0 20160202 for 8808c00000.display on minor 2
[   13.340941] [drm] [nvidia-drm] [GPU ID 0x00020000] Invalid framebuffer console info
[   13.350038] nv_platform 8808c00000.display: [drm] Cannot find any crtc or sizes

custom board:

# dmesg | grep -i drm                                                                                     
[   14.070696] systemd[1]: Starting Load Kernel Module drm...
[   14.525650] systemd[1]: [email protected]: Deactivated successfully.
[   14.527076] systemd[1]: Finished Load Kernel Module drm.
[   18.212023] [drm] Initialized tegra 1.0.99 20120330 for 8181200000.host1x.drm on minor 0
[   21.480082] [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
[   22.690216] [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 1
[   22.690239] nvidia 0000:01:00.0: [drm] No compatible format found
[   22.690245] nvidia 0000:01:00.0: [drm] Cannot find any crtc or sizes

One obvious difference is that my custom board is headless and have no display interface. Any ideas around this issue and how to get `nvargus_daemon`running are welcome!

test_egl.dev-kit.log (239.9 KB)

test_egl.custom.log (201.9 KB)

13

One step ahead. You must have a display configured in your device-tree even if it is a headless system. In my case I had to enable this in my dts:

        display@8808c00000 {
               status = "okay";

               nvdisplay-niso {
                       status = "okay";
               };
        };

This made a new dri device node called `dev/dri/card2` and with this my EGL test program works. I can also list my cameras in argus with `argus_oneshot -l`.

14

Below for headless referecing.

15

What kind of reference is this @ShaneCCC ? For me kind of irrelevant from what I found. The reference is talking about to disable the display node and I found I must have the display node enabled. Please clarify this.

16

Maybe it could be different for Thor(JP7)

Thanks for the information.