VIDEO NODE ACCESSING PROBLEM

[kamalesh1]

Hi,

I am using Jetpack 6.2.1. Orin NX with Jetson Nano super Developer Kit.

Jetpack - 5.1.2 → I have loaded the lt6911uxc non-i2c mipi driver which is in jetpack itself and the device tree. Finally video node created and i got the video output.

Jetpack - 6.2.1(L4T 36.4.4) → I searched the non-i2c lt6911uxc. But I got i2c lt6911uxc driver in nvidia-oot directory. In my case, I dont want i2c driver. So, I have taken the non-i2c lt6911uxc driver from jetapck 5.1.2 and loaded it in nvidia-oot directory itself. I have created the custom device tree and included this device tree in source/Linux_for_Tegra/source/hardware/nvidia/t23x/nv-public/nv-platform/tegra234-p3768-0000+p3767-xxxx-nv-common.dtsi.
Then I copied the modules from nvidia-oot and the image & dtbs from kernel-jammy-src to the original Linux_for_Tegra.

Finally Video node created.

But while accessing the video node with gst-launch-1.0. System goest to hang.

Note:- While giving fakesink with the gst-launch-1.0. After some debugging messaage, it is going to hang state. I attached the screen shot.

What could be the reason for this behaviour.

Regards,
Kamalesh

[carolyuu]

*** Please note that this reply is generated by LLM automatically ***

The issue you’re facing is related to the lt6911uxc non-i2c mipi driver on Jetpack 6.2.1 (L4T 36.4.4) on Jetson Orin NX. You’ve loaded the driver and created a custom device tree, but when accessing the video node with gst-launch-1.0, the system hangs.

Comparing your situation with the context provided, it seems that you’re experiencing a similar issue to the one described in the topic “Capturing Image from custom sensor fails with 4-csi lanes” by toho. In that case, the user was using a custom mipi-csi camera with 4 lanes on CSI-Port 0, and the system failed to capture images.

However, there are some differences between your situation and toho’s:

1. Driver version: You’re using the non-i2c lt6911uxc driver from Jetpack 5.1.2, whereas toho was using the EV9500M driver.
2. Device tree: You’ve created a custom device tree, whereas toho was using a pre-existing device tree.
3. System behavior: Your system hangs when accessing the video node, whereas toho’s system failed to capture images.

To troubleshoot your issue, you can try the following:

1. Verify the device tree: Double-check that your custom device tree is correct and properly configured for the lt6911uxc non-i2c mipi driver.
2. Check the driver version: Ensure that the driver version you’re using is compatible with Jetpack 6.2.1 (L4T 36.4.4).
3. Compare with a working configuration: Try to reproduce the working configuration from Jetpack 5.1.2 (L4T 35.2.1) on your Jetson Orin NX to see if the issue is specific to the newer Jetpack version.
4. Debugging: Analyze the system logs and debug messages to identify the cause of the hang. You can also try using a different gst-launch-1.0 command or a different video capture tool to see if the issue is specific to gst-launch-1.0.

By comparing your situation with similar issues and trying the suggested troubleshooting steps, you should be able to identify the root cause of the problem and find a solution.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

[kamalesh1]

Hi,

How to ensure the driver version or what are all the changes have to do?
In this jetpack only having i2c mipi driver.

Below i have given my customised device tree for this jetpack 6.2.1. Please correct me if i have any mistakes in this?

// SPDX-License-Identifier: GPL-2.0-only
// SPDX-FileCopyrightText: Copyright (c) 2023-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.

define CAM0_PWDN TEGRA234_MAIN_GPIO(H, 6)
define CAM_I2C_MUX TEGRA234_AON_GPIO(CC, 3)

include <dt-bindings/tegra234-p3767-0000-common.h>
include “dt-bindings/gpio/tegra234-gpio.h”

/ {
tegra-capture-vi {
num-channels = <1>;
ports {
#address-cells = <1>;
#size-cells = <0>;
vi_port0: port@0 {
reg = <0>;
lt6911uxc_vi_in0: endpoint {
port-index = <0>;
bus-width = <4>;
remote-endpoint = <&lt6911uxc_csi_out0>;
};
};
};
};

bus@0 {
host1x@13e00000 {
nvcsi@15a00000 {
num-channels = <1>;
#address-cells = <1>;
#size-cells = <0>;
csi_chan0: channel@0 {
reg = <0>;
ports {
#address-cells = <1>;
#size-cells = <0>;
csi_chan0_port0: port@0 {
reg = <0>;
lt6911uxc_csi_in0: endpoint@0 {
port-index = <0>;
bus-width = <4>;
remote-endpoint = <&lt6911uxc_out0>;
};
};
csi_chan0_port1: port@1 {
reg = <1>;
lt6911uxc_csi_out0: endpoint@1 {
remote-endpoint = <&lt6911uxc_vi_in0>;
};
};
};
};
};
};

cam_i2cmux {
    status = "okay";
    compatible = "i2c-mux-gpio";
    #address-cells = <1>;
    #size-cells = <0>;
    i2c-parent = <&cam_i2c>;
    mux-gpios = <&gpio_aon  CAM_I2C_MUX GPIO_ACTIVE_HIGH>;

    i2c@0 {
        status = "okay";
        reg = <0>;
        #address-cells = <1>;
        #size-cells = <0>;
        lt6911uxc@2d {
            status = "okay";
            compatible = "nvidia,lt6911uxc";
            reg = <0x2d>;
            devnode = "video0";
            physical_w = "4.713";
            physical_h = "3.494";
            sensor_model = "lt6911uxc";
            reset-gpios = <&gpio  CAM0_PWDN GPIO_ACTIVE_HIGH>;
            set_mode_delay_ms = "100";
            use_sensor_mode_id = "true";

            mode0 {
                mclk_khz = "24000";
                num_lanes = "4";
                tegra_sinterface = "serial_a";
                phy_mode = "DPHY";
                discontinuous_clk = "yes";
                dpcm_enable = "false";
                cil_settletime = "0";
                lane_polarity = "6";
                active_w = "1920";
                active_h = "1080";
                mode_type = "yuv";
                pixel_phase = "uyvy";
                csi_pixel_bit_depth = "16";
                readout_orientation = "0";
                line_length = "2000";
                inherent_gain = "1";
                mclk_multiplier = "24";
                pix_clk_hz = "148400000";
                gain_factor = "16";
                framerate_factor = "1000000";
                exposure_factor = "1000000";
                min_gain_val = "16";
                max_gain_val = "170";
                step_gain_val = "1";
                default_gain = "16";
                min_hdr_ratio = "1";
                max_hdr_ratio = "1";
                min_framerate = "2000000";
                max_framerate = "60000000";
                step_framerate = "1";
                default_framerate = "60000000";
                min_exp_time = "13";
                max_exp_time = "683709";
                step_exp_time = "1";
                default_exp_time = "16667";
                embedded_metadata_height = "0";
            };

            ports {
                #address-cells = <1>;
                #size-cells = <0>;
                port@0 {
                    reg = <0>;
                    lt6911uxc_out0: endpoint {
                        port-index = <0>;
                        bus-width = <4>;
                        remote-endpoint = <&lt6911uxc_csi_in0>;
                    };
                };
            };
        };
    };
  };
};

gpio@6000d000 {
    camera-control-output-low {
        gpio-hog;
        output-low;
        gpios = < CAM0_PWDN 0>;
        label = "cam0-pwdn";
    };
};

tcp: tegra-camera-platform {
    compatible = "nvidia,tegra-camera-platform";
    num_csi_lanes = <4>;
    max_lane_speed = <1500000>;
    min_bits_per_pixel = <10>;
    vi_peak_byte_per_pixel = <2>;
    vi_bw_margin_pct = <25>;
    max_pixel_rate = <240000>;
    isp_peak_byte_per_pixel = <5>;
    isp_bw_margin_pct = <25>;

    modules {
        cam_module0: module0 {
            badge = "LT6911uxc_front";
            position = "front";
            orientation = "1";
            cam_module0_drivernode0: drivernode0 {
                pcl_id = "v4l2_sensor";
                devname = "lt6911uxc 7-002d";
                sysfs-device-tree = "/sys/firmware/devicetree/base/bus@0/cam_i2cmux/i2c@0/lt6911uxc@2d";
            };
        };
    };
};

};

Is my device tree loading is correct?

Regards,
Kamalesh

[DaneLLL]

Hi,
We would suggest add debug prints in the driver code to identify where it hangs. And add debug prints to ensure the device tree is correctly parsed.

[kamalesh1]

Hi DaneLLL,

I have taken some debug messages. After driver loading, video node created. But while accessing the node via gstreamer and opencv, the screen got strucked. It is closed until restart. so, I could not take the debug after running.

sudo dmesg | grep -i lt6911uxc
[ 8.451059] lt6911uxc 9-002d: probing lt6911uxc v4l2 sensor at addr 0x2d
[ 8.451209] lt6911uxc 9-002d: tegracam sensor driver:lt6911uxc_v2.0.6
[ 8.454470] tegra-camrtc-capture-vi tegra-capture-vi: subdev lt6911uxc 9-002d bound
[ 8.480994] lt6911uxc 9-002d: detected lt6911uxc sensor

media-ctl -p /dev/video0
Media controller API version 5.15.148

Media device information

driver tegra-camrtc-ca
model NVIDIA Tegra Video Input Device
serial
bus info
hw revision 0x3
driver version 5.15.148

Device topology

• entity 1: 13e00000.host1x:nvcsi@15a00000- (2 pads, 2 links)
  type V4L2 subdev subtype Unknown flags 0
  device node name /dev/v4l-subdev0
  pad0: Sink
  ← “lt6911uxc 9-002d”:0 [ENABLED]
  pad1: Source
  → “vi-output, lt6911uxc 9-002d”:0 [ENABLED]

• entity 4: lt6911uxc 9-002d (1 pad, 1 link)
  type V4L2 subdev subtype Sensor flags 0
  device node name /dev/v4l-subdev1
  pad0: Source
  [fmt:UYVY8_1X16/1920x1080@1/60 field:none colorspace:srgb]
  → “13e00000.host1x:nvcsi@15a00000-”:0 [ENABLED]

• entity 6: vi-output, lt6911uxc 9-002d (1 pad, 1 link)
  type Node subtype V4L flags 0
  device node name /dev/video0
  pad0: Sink
  ← “13e00000.host1x:nvcsi@15a00000-”:1 [ENABLED]

v4l2-ctl -d /dev/video0 --all
Driver Info:
Driver name : tegra-video
Card type : vi-output, lt6911uxc 9-002d
Bus info : platform:tegra-capture-vi:0
Driver version : 5.15.148
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : tegra-camrtc-ca
Model : NVIDIA Tegra Video Input Device
Serial :
Bus info :
Media version : 5.15.148
Hardware revision: 0x00000003 (3)
Driver version : 5.15.148
Interface Info:
ID : 0x03000008
Type : V4L Video
Entity Info:
ID : 0x00000006 (6)
Name : vi-output, lt6911uxc 9-002d
Function : V4L2 I/O
Pad 0x01000007 : 0: Sink
Link 0x0200000c: from remote pad 0x1000003 of entity ‘13e00000.host1x:nvcsi@15a00000-’ (Unknown sub-device (0002000a)): Data, Enabled
Priority: 2
Video input : 0 (Camera 0: no power)
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : ‘UYVY’ (UYVY 4:2:2)
Field : None
Bytes per Line : 3840
Size Image : 4147200
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: 60.000 (60/1)
Read buffers : 0

Camera Controls

       sensor_configuration 0x009a2032 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[22] flags=read-only, volatile, has-payload
     sensor_mode_i2c_packet 0x009a2033 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[1026] flags=read-only, volatile, has-payload
  sensor_control_i2c_packet 0x009a2034 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[1026] flags=read-only, volatile, has-payload
                bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
			0: 0 (0x0)
			1: 1 (0x1)
            override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
			0: 0 (0x0)
			1: 1 (0x1)
               height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                 size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0 (1 0x1)
			0: 1 (0x1)
			1: 65536 (0x10000)
			2: 131072 (0x20000)
           write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
   sensor_signal_properties 0x009a2069 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][18] flags=read-only, has-payload
    sensor_image_properties 0x009a206a (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
  sensor_control_properties 0x009a206b (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][36] flags=read-only, has-payload
          sensor_dv_timings 0x009a206c (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
           low_latency_mode 0x009a206d (bool)   : default=0 value=0
           preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
override_capture_timeout_ms 0x009a206f (int)    : min=-1 max=2147483647 step=1 default=2500 value=2500
               sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=1 flags=read-only

I attached the screen after running the gstreamer with imagesink. What may be the problem here?

Or Is there any method to debug?

Thank you

Regards,
Kamalesh

[kamalesh1]

Hi,

Is my device tree and my uploading path is correct?

Please help me to move forward.

Regards,
Kamalesh

[DaneLLL]

Hi,
Please try v4l2-ctl command and see if you can capture frame data. If it fails, it means camera driver is not ready. Please refer to camera sensor driver programming guide to check further.

Sensor Software Driver Programming — NVIDIA Jetson Linux Developer Guide