Can't start YOLOv8 detection with custom model

natanel · December 14, 2023, 8:40pm

I’m trying to run the YOLOv8 detection node on a Jetson AGX Orin with a custom model I made in RoboFlow. for some reason it stops without giving a detailed error, and I can’t understand why. When I do the same with the “yolov8s.pt” model from the example it works fine.

I tried exporting the model on several computers, including the Jetson itself, but I got the same result. I compared the models in netron.app and the only difference I can see is the output size, which is 1x84x8400 for the default model and 1x6x8400 for my custom one, but I don’t know if that could be the issue. What else can I try?

Here are the logs for each model, they also look almost identical except for the crash with my model:

yolov8s.onnx:

$ ros2 launch isaac_ros_yolov8 isaac_ros_yolov8_visualize.launch.py model_file_path:=/tmp/yolov8s.onnx engine_file_path:=/tmp/yolov8s.plan input_binding_names:=['images'] output_binding_names:=['output0'] network_image_width:=640 network_image_height:=640 force_engine_update:=False image_mean:=[0.0,0.0,0.0] image_stddev:=[1.0,1.0,1.0] input_image_width:=640 input_image_height:=640 confidence_threshold:=0.25 nms_threshold:=0.45
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-12-14-22-15-42-085841-rpagx-desktop-8604
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [8616]
[INFO] [isaac_ros_yolov8_visualizer.py-2]: process started with pid [8618]
[INFO] [rqt_image_view-3]: process started with pid [8620]
[component_container_mt-1] [INFO] [1702584942.987695882] [tensor_rt_container]: Load Library: /opt/ros/humble/lib/libdnn_image_encoder_node.so
[component_container_mt-1] [INFO] [1702584943.026007092] [tensor_rt_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode>
[component_container_mt-1] [INFO] [1702584943.026115925] [tensor_rt_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode>
[component_container_mt-1] [INFO] [1702584943.037648027] [NitrosContext]: [NitrosContext] Creating a new shared context
[component_container_mt-1] [INFO] [1702584943.037876764] [dnn_image_encoder]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1702584943.039109220] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1702584943.046630674] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container_mt-1] [INFO] [1702584943.051038796] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_sight.so
[component_container_mt-1] [INFO] [1702584943.055852970] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_atlas.so
[component_container_mt-1] [INFO] [1702584943.061101930] [NitrosContext]: [NitrosContext] Loading application: '/workspaces/isaac_ros-dev/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_graph.yaml'
[component_container_mt-1] [INFO] [1702584943.062410642] [NitrosContext]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1702584943.066421898] [NitrosContext]: [NitrosContext] Running application...
[component_container_mt-1] e[33m2023-12-14 22:15:43.066 WARN  gxf/std/program.cpp@514: No system specified. Nothing to doe[0m
[component_container_mt-1] [INFO] [1702584943.070031040] [dnn_image_encoder]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1702584943.070098337] [dnn_image_encoder]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1702584943.073301492] [dnn_image_encoder]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1702584943.073389717] [dnn_image_encoder]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1702584943.075985765] [dnn_image_encoder]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1702584943.076058405] [dnn_image_encoder]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1702584943.076576424] [dnn_image_encoder]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1702584943.077281932] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so
[component_container_mt-1] [INFO] [1702584943.079826716] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1702584943.081695335] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1702584943.086328580] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so
[component_container_mt-1] [INFO] [1702584943.094860664] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1702584943.096391233] [dnn_image_encoder]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1702584943.103743822] [dnn_image_encoder]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1702584943.352665947] [dnn_image_encoder]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1702584943.360597036] [dnn_image_encoder]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1702584943.387506032] [dnn_image_encoder]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/dnn_image_encoder' in container '/tensor_rt_container'
[component_container_mt-1] [INFO] [1702584943.396611527] [tensor_rt_container]: Load Library: /opt/ros/humble/lib/libtensor_rt_node.so
[component_container_mt-1] [INFO] [1702584943.501672424] [tensor_rt_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TensorRTNode>
[component_container_mt-1] [INFO] [1702584943.501767816] [tensor_rt_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TensorRTNode>
[component_container_mt-1] [INFO] [1702584943.547540128] [tensor_rt]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1702584943.565932112] [tensor_rt]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1702584943.566041232] [tensor_rt]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1702584943.573667487] [tensor_rt]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1702584943.573768159] [tensor_rt]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1702584943.574401283] [tensor_rt]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1702584943.574501508] [tensor_rt]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1702584943.574752613] [tensor_rt]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1702584943.576801042] [tensor_rt]: [NitrosContext] Loading extension: gxf/tensor_rt/libgxf_tensor_rt.so
[component_container_mt-1] [INFO] [1702584943.579409602] [tensor_rt]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1702584943.581675792] [tensor_rt]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1702584943.598012467] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584943.600504962] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584943.609254424] [tensor_rt]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1702584943.610883170] [tensor_rt]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1702584943.611080995] [tensor_rt]: [NitrosPublisherSubscriberGroup] Pinning the component "inference/rx" (type="nvidia::gxf::DoubleBufferReceiver") to use its compatible format only: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1702584943.611795687] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584943.613559570] [tensor_rt]: [NitrosPublisherSubscriberGroup] Pinning the component "sink/sink" (type="nvidia::isaac_ros::MessageRelay") to use its compatible format only: "nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1702584943.623984978] [tensor_rt]: [NitrosNode] Starting negotiation...
[component_container_mt-1] [INFO] [1702584943.627401062] [dnn_image_encoder]: Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/tensor_rt' in container '/tensor_rt_container'
[component_container_mt-1] [INFO] [1702584943.631565088] [tensor_rt_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_yolov8/lib/libyolov8_decoder_node.so
[component_container_mt-1] [INFO] [1702584943.645461461] [tensor_rt_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::yolov8::YoloV8DecoderNode>
[component_container_mt-1] [INFO] [1702584943.645561685] [tensor_rt_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::yolov8::YoloV8DecoderNode>
[component_container_mt-1] [INFO] [1702584943.712885840] [yolov8_decoder_node.ManagedNitrosSubscriber]: Starting Managed Nitros Subscriber
[component_container_mt-1] [INFO] [1702584943.713171985] [tensor_rt]: Negotiating
[component_container_mt-1] [INFO] [1702584943.713222418] [tensor_rt]: Could not negotiate
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/yolov8_decoder_node' in container '/tensor_rt_container'
[component_container_mt-1] [INFO] [1702584943.732910122] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584943.812029036] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584943.912524337] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584944.012497170] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584944.111987217] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584944.212646583] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584944.323629851] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702584944.390573939] [dnn_image_encoder]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1702584944.390675764] [dnn_image_encoder]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1702584944.390707124] [dnn_image_encoder]: [NitrosPublisher] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1702584944.390725268] [dnn_image_encoder]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1702584944.390736276] [dnn_image_encoder]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/image", data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1702584944.391097719] [dnn_image_encoder]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1702584944.604689005] [dnn_image_encoder]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/NXHQXLCPUT/NXHQXLCPUT.yaml"
[component_container_mt-1] [INFO] [1702584944.604837294] [dnn_image_encoder]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1702584944.604863662] [dnn_image_encoder]: In DNN Image Encoder Node preLoadGraphCallback().
[component_container_mt-1] [INFO] [1702584944.604915950] [dnn_image_encoder]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1702584944.604935566] [dnn_image_encoder]: [NitrosContext] Loading application: '/tmp/isaac_ros_nitros/graphs/NXHQXLCPUT/NXHQXLCPUT.yaml'
[component_container_mt-1] [INFO] [1702584944.622488889] [dnn_image_encoder]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1702584944.625850670] [dnn_image_encoder]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1702584944.626597170] [dnn_image_encoder]: In DNN Image Encoder Node postLoadGraphCallback().
[component_container_mt-1] [INFO] [1702584944.627268119] [dnn_image_encoder]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1702584944.628579295] [tensor_rt]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1702584944.628654655] [tensor_rt]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1702584944.628674399] [tensor_rt]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1702584944.628688607] [tensor_rt]: [NitrosPublisher] Use only the compatible publisher: topic_name="/tensor_sub", data_format="nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1702584944.628703071] [tensor_rt]: [NitrosSubscriber] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1702584944.628971937] [tensor_rt]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1702584944.654464380] [tensor_rt]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/PBDGUHNPLS/PBDGUHNPLS.yaml"
[component_container_mt-1] [INFO] [1702584944.654589117] [tensor_rt]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1702584944.654608285] [tensor_rt]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1702584944.713089378] [dnn_image_encoder]: [NitrosContext] Running application...
[component_container_mt-1] [INFO] [1702584944.720962386] [dnn_image_encoder]: [NitrosNode] Starting a heartbeat timer (eid=65)
[component_container_mt-1] [INFO] [1702584944.721025266] [tensor_rt]: [NitrosContext] Loading application: '/tmp/isaac_ros_nitros/graphs/PBDGUHNPLS/PBDGUHNPLS.yaml'
[component_container_mt-1] [INFO] [1702584944.725282284] [tensor_rt]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1702584944.725447501] [tensor_rt]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1702584944.725477165] [tensor_rt]: In TensorRTNode postLoadGraphCallback().
[component_container_mt-1] e[33m2023-12-14 22:15:44.724 WARN  gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'dummy_rx' in component ''.e[0m
[component_container_mt-1] e[33m2023-12-14 22:15:44.725 WARN  gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'dev_id' in component 'stream'.e[0m
[component_container_mt-1] [INFO] [1702584947.829563962] [tensor_rt]: Read tensor shape information from TRT Model Engine: /tmp/yolov8s.plan
[component_container_mt-1] [INFO] [1702584947.829813660] [tensor_rt]: Tensors 2822400 bytes, num outputs 40 x tensors per output 1 = 40 blocks
[component_container_mt-1] [INFO] [1702584947.830080893] [tensor_rt]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1702584947.851747265] [tensor_rt]: [NitrosContext] Running application...
[component_container_mt-1] [INFO] [1702584947.854254513] [tensor_rt]: [NitrosNode] Starting a heartbeat timer (eid=150)
[component_container_mt-1] e[33m2023-12-14 22:15:47.892 WARN  /data/jenkins/ros/workspace/ly_debian-compatible-release-2.1/isaac_ros-dev/ros_ws/src/isaac_ros_dnn_inference/isaac_ros_tensor_rt/gxf/tensor_rt/tensor_rt_inference.cpp@152: TRT WARNING: Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.

custom.onnx:

$ ros2 launch isaac_ros_yolov8 isaac_ros_yolov8_visualize.launch.py model_file_path:=/tmp/custom.onnx engine_file_path:=/tmp/custom.plan input_binding_names:=['images'] output_binding_names:=['output0'] network_image_width:=640 network_image_height:=640 force_engine_update:=False image_mean:=[0.0,0.0,0.0] image_stddev:=[1.0,1.0,1.0] input_image_width:=640 input_image_height:=640 confidence_threshold:=0.25 nms_threshold:=0.45
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-12-14-21-55-18-882054-rpagx-desktop-8447
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [8459]
[INFO] [isaac_ros_yolov8_visualizer.py-2]: process started with pid [8461]
[INFO] [rqt_image_view-3]: process started with pid [8463]
[component_container_mt-1] [INFO] [1702583719.776698750] [tensor_rt_container]: Load Library: /opt/ros/humble/lib/libdnn_image_encoder_node.so
[component_container_mt-1] [INFO] [1702583719.815747221] [tensor_rt_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode>
[component_container_mt-1] [INFO] [1702583719.815852630] [tensor_rt_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode>
[component_container_mt-1] [INFO] [1702583719.830300946] [NitrosContext]: [NitrosContext] Creating a new shared context
[component_container_mt-1] [INFO] [1702583719.830510003] [dnn_image_encoder]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1702583719.831722139] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1702583719.839314795] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container_mt-1] [INFO] [1702583719.843605574] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_sight.so
[component_container_mt-1] [INFO] [1702583719.848256323] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_atlas.so
[component_container_mt-1] [INFO] [1702583719.853330531] [NitrosContext]: [NitrosContext] Loading application: '/workspaces/isaac_ros-dev/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_graph.yaml'
[component_container_mt-1] [INFO] [1702583719.854602091] [NitrosContext]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1702583719.858535940] [NitrosContext]: [NitrosContext] Running application...
[component_container_mt-1] e[33m2023-12-14 21:55:19.858 WARN  gxf/std/program.cpp@514: No system specified. Nothing to doe[0m
[component_container_mt-1] [INFO] [1702583719.862045531] [dnn_image_encoder]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1702583719.862108667] [dnn_image_encoder]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1702583719.865183534] [dnn_image_encoder]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1702583719.865257903] [dnn_image_encoder]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1702583719.867973440] [dnn_image_encoder]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1702583719.868047393] [dnn_image_encoder]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1702583719.868573092] [dnn_image_encoder]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1702583719.869310921] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so
[component_container_mt-1] [INFO] [1702583719.871672823] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1702583719.873388258] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1702583719.878101824] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so
[component_container_mt-1] [INFO] [1702583719.886888120] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1702583719.888327425] [dnn_image_encoder]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1702583719.895700144] [dnn_image_encoder]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1702583720.140162715] [dnn_image_encoder]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1702583720.157756490] [dnn_image_encoder]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1702583720.209387185] [dnn_image_encoder]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/dnn_image_encoder' in container '/tensor_rt_container'
[component_container_mt-1] [INFO] [1702583720.216998721] [tensor_rt_container]: Load Library: /opt/ros/humble/lib/libtensor_rt_node.so
[component_container_mt-1] [INFO] [1702583720.301316439] [tensor_rt_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TensorRTNode>
[component_container_mt-1] [INFO] [1702583720.301402935] [tensor_rt_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TensorRTNode>
[component_container_mt-1] [INFO] [1702583720.363875203] [tensor_rt]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1702583720.379492454] [tensor_rt]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1702583720.379566246] [tensor_rt]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1702583720.387947899] [tensor_rt]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1702583720.388044476] [tensor_rt]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1702583720.388688192] [tensor_rt]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1702583720.388780705] [tensor_rt]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1702583720.389038082] [tensor_rt]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1702583720.395725516] [tensor_rt]: [NitrosContext] Loading extension: gxf/tensor_rt/libgxf_tensor_rt.so
[component_container_mt-1] [INFO] [1702583720.398369405] [tensor_rt]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1702583720.400628203] [tensor_rt]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1702583720.401526449] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583720.404108418] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583720.430902699] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583720.460441766] [tensor_rt]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1702583720.474878913] [tensor_rt]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1702583720.475021026] [tensor_rt]: [NitrosPublisherSubscriberGroup] Pinning the component "inference/rx" (type="nvidia::gxf::DoubleBufferReceiver") to use its compatible format only: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1702583720.475074595] [tensor_rt]: [NitrosPublisherSubscriberGroup] Pinning the component "sink/sink" (type="nvidia::isaac_ros::MessageRelay") to use its compatible format only: "nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1702583720.501531242] [tensor_rt]: [NitrosNode] Starting negotiation...
[component_container_mt-1] [INFO] [1702583720.505920646] [dnn_image_encoder]: Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/tensor_rt' in container '/tensor_rt_container'
[component_container_mt-1] [INFO] [1702583720.509471900] [tensor_rt_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_yolov8/lib/libyolov8_decoder_node.so
[component_container_mt-1] [INFO] [1702583720.527656560] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583720.530451137] [tensor_rt_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::yolov8::YoloV8DecoderNode>
[component_container_mt-1] [INFO] [1702583720.530520002] [tensor_rt_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::yolov8::YoloV8DecoderNode>
[component_container_mt-1] [INFO] [1702583720.599030547] [yolov8_decoder_node.ManagedNitrosSubscriber]: Starting Managed Nitros Subscriber
[component_container_mt-1] [INFO] [1702583720.599129844] [tensor_rt]: Negotiating
[component_container_mt-1] [INFO] [1702583720.599188980] [tensor_rt]: Could not negotiate
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/yolov8_decoder_node' in container '/tensor_rt_container'
[component_container_mt-1] [INFO] [1702583720.628867824] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583720.747978466] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583720.842384504] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583720.955653028] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583721.047891852] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583721.160328660] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1702583721.211474328] [dnn_image_encoder]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1702583721.211567544] [dnn_image_encoder]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1702583721.211597240] [dnn_image_encoder]: [NitrosPublisher] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1702583721.211615224] [dnn_image_encoder]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1702583721.211626329] [dnn_image_encoder]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/image", data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1702583721.211988891] [dnn_image_encoder]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1702583721.362736085] [dnn_image_encoder]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/PGBADJMFGK/PGBADJMFGK.yaml"
[component_container_mt-1] [INFO] [1702583721.362882486] [dnn_image_encoder]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1702583721.362912374] [dnn_image_encoder]: In DNN Image Encoder Node preLoadGraphCallback().
[component_container_mt-1] [INFO] [1702583721.362964310] [dnn_image_encoder]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1702583721.362983286] [dnn_image_encoder]: [NitrosContext] Loading application: '/tmp/isaac_ros_nitros/graphs/PGBADJMFGK/PGBADJMFGK.yaml'
[component_container_mt-1] [INFO] [1702583721.381701389] [dnn_image_encoder]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1702583721.388748794] [dnn_image_encoder]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1702583721.388795866] [dnn_image_encoder]: In DNN Image Encoder Node postLoadGraphCallback().
[component_container_mt-1] [INFO] [1702583721.389006107] [dnn_image_encoder]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1702583721.469970716] [dnn_image_encoder]: [NitrosContext] Running application...
[component_container_mt-1] [INFO] [1702583721.470436223] [dnn_image_encoder]: [NitrosNode] Starting a heartbeat timer (eid=65)
[component_container_mt-1] [INFO] [1702583721.505847967] [tensor_rt]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1702583721.505961951] [tensor_rt]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1702583721.505982015] [tensor_rt]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1702583721.505998752] [tensor_rt]: [NitrosPublisher] Use only the compatible publisher: topic_name="/tensor_sub", data_format="nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1702583721.506013440] [tensor_rt]: [NitrosSubscriber] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1702583721.506279425] [tensor_rt]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1702583721.511626467] [tensor_rt]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/RZSWRLWKET/RZSWRLWKET.yaml"
[component_container_mt-1] [INFO] [1702583721.511685572] [tensor_rt]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1702583721.511702724] [tensor_rt]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1702583721.511721156] [tensor_rt]: [NitrosContext] Loading application: '/tmp/isaac_ros_nitros/graphs/RZSWRLWKET/RZSWRLWKET.yaml'
[component_container_mt-1] e[33m2023-12-14 21:55:21.516 WARN  gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'dummy_rx' in component ''.e[0m
[component_container_mt-1] e[33m2023-12-14 21:55:21.518 WARN  gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'dev_id' in component 'stream'.e[0m
[component_container_mt-1] [INFO] [1702583721.518877553] [tensor_rt]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1702583721.519483317] [tensor_rt]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1702583721.519923192] [tensor_rt]: In TensorRTNode postLoadGraphCallback().
[component_container_mt-1] [INFO] [1702583725.206943899] [tensor_rt]: Read tensor shape information from TRT Model Engine: /tmp/best.plan
[component_container_mt-1] [INFO] [1702583725.207203293] [tensor_rt]: Tensors 201600 bytes, num outputs 40 x tensors per output 1 = 40 blocks
[component_container_mt-1] [INFO] [1702583725.207464606] [tensor_rt]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1702583725.225039630] [tensor_rt]: [NitrosContext] Running application...
[component_container_mt-1] [INFO] [1702583725.225320463] [tensor_rt]: [NitrosNode] Starting a heartbeat timer (eid=184)
[ERROR] [component_container_mt-1]: process has died [pid 8459, exit code -11, cmd '/opt/ros/humble/lib/rclcpp_components/component_container_mt --ros-args --log-level INFO --ros-args -r __node:=tensor_rt_container -r __ns:=/'].

Thank you in advance.

Nikolay196 · December 15, 2023, 11:16am

Hi @natanel , I had a similar problem, how many detection classes do you have in the model?

There was a one class in my model.

It helped me in the file:
/workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_yolov8/src/yolov8_decoder_node.cpp
On line 76, change the number of classes to 2

yolov8_decoder_node.cpp

  75  //  Output dimensions = [1, 84, 8400]
  76  int num_classes = 2;
  77  int out_dim = 8400;

And change /workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_yolov8/scripts/isaac_ros_yolov8_visualizer copy.py
On:

isaac_ros_yolov8_visualizer.py

 #!/usr/bin/env python3

# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0

# This script listens for images and object detections on the image,
# then renders the output boxes on top of the image and publishes
# the result as an image message

import cv2
import cv_bridge
import message_filters
import rclpy
from rclpy.node import Node
from sensor_msgs.msg import Image
from vision_msgs.msg import Detection2DArray

names = {
        0: <your class>,
}


class Yolov8Visualizer(Node):
    QUEUE_SIZE = 10
    color = (0, 255, 0)
    bbox_thickness = 2

    def __init__(self):
        super().__init__('yolov8_visualizer')
        self._bridge = cv_bridge.CvBridge()
        self._processed_image_pub = self.create_publisher(
            Image, 'yolov8_processed_image',  self.QUEUE_SIZE)

        self._detections_subscription = message_filters.Subscriber(
            self,
            Detection2DArray,
            'detections_output')
        self._image_subscription = message_filters.Subscriber(
            self,
            Image,
            'image')

        self.time_synchronizer = message_filters.TimeSynchronizer(
            [self._detections_subscription, self._image_subscription],
            self.QUEUE_SIZE)

        self.time_synchronizer.registerCallback(self.detections_callback)

    def detections_callback(self, detections_msg, img_msg):
        txt_color = (255, 0, 255)
        cv2_img = self._bridge.imgmsg_to_cv2(img_msg)
        

        for detection in detections_msg.detections:
            center_x = detection.bbox.center.position.x
            center_y = detection.bbox.center.position.y

            #center_y = center_y * 0.75

            width = detection.bbox.size_x
            height = detection.bbox.size_y

            #height = height * 0.75

            label = names[int(detection.results[0].hypothesis.class_id)]
            conf_score = detection.results[0].hypothesis.score
            label = f'{label} {conf_score:.2f}'

            min_pt = (round(center_x - (width / 2.0)),
                      round(center_y - (height / 2.0)))
            max_pt = (round(center_x + (width / 2.0)),
                      round(center_y + (height / 2.0)))

            lw = max(round((img_msg.height + img_msg.width) / 2 * 0.003), 2)  # line width
            tf = max(lw - 1, 1)  # font thickness
            # text width, height
            w, h = cv2.getTextSize(label, 0, fontScale=lw / 3, thickness=tf)[0]
            outside = min_pt[1] - h >= 3

            cv2.rectangle(cv2_img, min_pt, max_pt,
                          (255, 100, 255), self.bbox_thickness)


            #min_pt = (min_pt[0], round(min_pt[1] * 0.7))
            #max_pt = (max_pt[0], round(max_pt[1] * 0.7))

            cv2.rectangle(cv2_img, min_pt, max_pt,
                          self.color, self.bbox_thickness)
            #cv2.putText(cv2_img, label, (min_pt[0], min_pt[1]-2 if outside else min_pt[1]+h+2),
            #            0, lw / 3, txt_color, thickness=tf, lineType=cv2.LINE_AA)
        
        #cv2_img = cv2.resize(cv2_img, (640, 480))

        processed_img = self._bridge.cv2_to_imgmsg(
            cv2_img, encoding=img_msg.encoding)
        self._processed_image_pub.publish(processed_img)


def main():
    rclpy.init()
    rclpy.spin(Yolov8Visualizer())
    rclpy.shutdown()


if __name__ == '__main__':
    main()

And after that of course:

colcon build --packages-select isaac_ros_yolov8

Of course this is a temporary solution, there is probably a better way to do this.

natanel · December 15, 2023, 1:02pm

Hey @Nikolay196 looks like it worked, thank you very much!

system · December 29, 2023, 1:03pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Cannot start yolov8 detection Isaac ROS isaac-ros-object-detection	10	1224	December 29, 2023
Can't start isaac ros yolov8 with Quickstart manual Isaac ROS isaac-ros-object-detection	4	521	April 2, 2024
Running my custom trained yolo v8n model on ROS (on Jetson Orin nano) Jetson Orin Nano ros	4	2325	July 17, 2023
Use custom RT-DETR model for FoundationPose Isaac ROS tensorrt , cuda	5	654	December 12, 2025
DeepstreamSDK 4.0.1 Custom Yolov3-Tiny Error DeepStream SDK	10	1582	April 27, 2020
Nvidia isaac ros with yolov8 Isaac ROS	4	1059	October 22, 2023
Yolov8 implementation on foundationpose package using realsense camera Isaac ROS camera , isaac-ros-object-detection , isaac-ros-pose-estimation	0	354	November 21, 2024
yolo custom model file and error from primary-nvinference-engine DeepStream SDK	15	3036	October 12, 2021
Isaac_ros_rtdetr cannot detect targets using custom model Isaac Gym tensorrt	2	133	March 25, 2025
No inference results for custom object using isaac_ros_centerpose - Compression requires single-channel 32bit-floating point or 16bit raw depth images Isaac ROS	2	92	February 12, 2025

Can't start YOLOv8 detection with custom model

Related topics