How to initialize a video encoder on Jetson Thor？

[deeptalkcamera]

We have successfully initialized the video encoder on Jetson orin using the following Topic patch:r35_31_add_nvvideoencoder.zip

How to use v4l2 to capture videos in Jetson Orin r35 Jetpack 5.0 and encode them using a hardware encoding chip

Continuing the discussion from tegra multimedia samples not working properly: My machine is Jetson Orin Jetpack 5.0.2 L4T: 35.1.0 I referred to the code provided by Nvidia，01_video_encode (video encode) and 12_camera_v4l2_cuda (camera capture CUDA processing) But it didn’t work out Do you have R35 video capture encoding example code

However, the video encoder seems to have undergone some changes on the Thor, with the introduction of isGPUEnabled(), complicating its use.
Could you explain the benefits of including a GPU? And provide sample code for initializing the video encoder, similar to the one in r35_31_add_nvvideoencoder.zip:

static bool
encoder_initialize(context_t * ctx)
{
    ctx->enc = NvVideoEncoder::createVideoEncoder("enc0");
    if (ctx->enc == NULL)
        ERROR_RETURN("Failed to create video encoder");

    if (ctx->enc->setCapturePlaneFormat(V4L2_PIX_FMT_H264, ctx->cam_w,
                ctx->cam_h, 2 * 1024 * 1024) < 0)
        ERROR_RETURN("Failed to set up ENC capture plane format");

    if (ctx->enc->setOutputPlaneFormat(V4L2_PIX_FMT_YUV420M, ctx->cam_w,
                ctx->cam_h) < 0)
        ERROR_RETURN("Failed to set up ENC output plane format");

    if (ctx->enc->setBitrate(4<<20) < 0)
        ERROR_RETURN("Failed to set up ENC bitrate");

    if (ctx->enc->setProfile(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) < 0)
        ERROR_RETURN("Failed to set up ENC profile");

    if (ctx->enc->setLevel(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) < 0)
        ERROR_RETURN("Failed to set up ENC level");

    if (ctx->enc->setRateControlMode(V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) < 0)
        ERROR_RETURN("Failed to set up ENC rate control mode");

    if (ctx->enc->output_plane.reqbufs(V4L2_MEMORY_DMABUF, OUTPLANE_BUFNUM))
        ERROR_RETURN("Failed to set up ENC output plane");

    if (ctx->enc->capture_plane.setupPlane(V4L2_MEMORY_MMAP, 6, true, false) < 0)
        ERROR_RETURN("Failed to set up ENC capture plane");

    ctx->enc->subscribeEvent(V4L2_EVENT_EOS,0,0);

    return true;
}

[whitesscott]

This might show something.

/usr/src/jetson_multimedia_api/

/usr/src/jetson_sipl_api/sipl

[DaneLLL]

Hi,
We move to Video CODEC SDK on AGX Thor and isGPUEnabled() is true. Have you seen any error after applying the patch to Jetpack 7? It is supposed to work for basic encoding.

[deeptalkcamera]

Our camera data format is YUV422 UYVY, which NVENC does not support. Therefore, we designed a method to use VIC to convert the format to YUV420 and then use NVENC for encoding.

After running the above process with offline UYVY data, the encoded H264 data frames display abnormally. I’m wondering if there’s a problem with the encoder configuration. Could you please help me identify possible causes?

The encoder configuration and calling logic are as follows:

for_nv_1020.zip (4.4 MB)

[DavidDDD]

Hi,

Thank you for your feedback.
We have updated the nvvideoencoder patch as shown in the attached file.

md5sum r38_2_add_nvvideoencoder.patch 
5bb733564493db1f0bf1a97d5fa3a56d  r38_2_add_nvvideoencoder.patch

r38_2_add_nvvideoencoder.zip (3.6 KB)

After reviewing the zip file you provided, please try the patch below and confirm if it works correctly.

diff --git a/video_encoder.cc b/video_encoder.cc
--- a/video_encoder.cc
+++ b/video_encoder.cc
@@ -63,7 +63,7 @@ int VideoEncoder::initialize(EncodeContext &ctx) {
     }
 
     if (m_videoEncoder->isGPUEnabled()) {
+        if (m_videoEncoder->setOutputPlaneFormat(V4L2_PIX_FMT_YUV420M, ctx.width_src,
-        if (m_videoEncoder->setOutputPlaneFormat(V4L2_PIX_FMT_NV12M, ctx.width_src,
                 ctx.height_src, V4L2_COLORSPACE_SMPTE170M, false) < 0) {
             syslog(LOG_ERR, "%s VideoEncoder setOutputPlaneFormat failed.\n", __func__);
             return -1;

Thanks

[deeptalkcamera]

Thank you for your review. It was indeed an issue with my setOutputPlaneFormat() method. After making these changes, my local code is now working correctly.

I’ll try r38_2_add_nvvideoencoder.zip

[deeptalkcamera]

Hi DavidDDD:

My local code runs successfully, but the output h264 has an issue during playback:

As you can see, when a fast-moving object passes by, some ghosting appears in the image.

I also tested the r38_2_add_nvvideoencoder.zip patch you provided. After fixing the CUDA mapping issue (this is another fixed topic), it encoded and displayed correctly.

patch_and_h264output.zip (12.9 MB)

I saved the encoded result, and the same ghosting issue occurs during playback. Could you please take a look at this phenomenon and tell me what’s causing it?

Also, if I decode the h264, will this ghosting affect the decoding result?

[deeptalkcamera]

Considering this ghosting issue might be a different issue, let’s use a new Topic track.

NVENC video encoder output h264 ghosting issue

The question stems from this topic: How to initialize a video encoder on Jetson Thor？ - #10 by deeptalkcamera Using r38_2_add_nvvideoencoder.zip for H.264 encoding and saving the encoded result, some ghosting can be seen on the screen during playback. [image] My code modifications to r38_2_add_nvvideoencoder.zip are as follows: patch_and_h264output.zip (12.9 MB) Our camera data format is YUV422 UYVY