Use custom RT-DETR model for FoundationPose

Robotics & Edge Computing Robotics - Isaac Isaac ROS
,
1

Hi,
System Specifications:

  • Operating System: Jetson AGX ORIN JetPack 6.0

I trained a rt-detr model follow the RT-DETR (Realtime Detection Transformer) - Ultralytics YOLO Docs, and covert it using yolo export model=./weights/rt-detr-l.pt format=onnx
then

 ros2 launch isaac_ros_examples isaac_ros_examples.launch.py \
 launch_fragments:=foundationpose \
 interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/quickstart_interface_specs.json \
 mesh_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/tx/my_tx.obj \
 texture_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/tx/my_tx.jpg \
 score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan \
 rt_detr_model_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rt-detr-l.onnx \
 refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rt-detr-l.plan

error

[component_container_mt-1] [INFO] [1724421025.299676381] [tensor_rt]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1724421025.304128165] [tensor_rt]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421025.304218080] [tensor_rt]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] 2024-08-23 21:50:25.304 ERROR gxf/std/program.cpp@551: Attempted interrupting when not running (state=0).
[component_container_mt-1] 2024-08-23 21:50:25.304 ERROR gxf/core/runtime.cpp@1468: Graph interrupt failed with error: GXF_INVALID_EXECUTION_SEQUENCE
[component_container_mt-1] [ERROR] [1724421025.304299292] [tensor_rt]: [NitrosContext] GxfGraphInterrupt Error: GXF_INVALID_EXECUTION_SEQUENCE
[component_container_mt-1] [INFO] [1724421025.304319643] [tensor_rt]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421025.304339610] [tensor_rt]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1724421025.304357401] [tensor_rt]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421025.304380536] [tensor_rt]: [NitrosNode] Application termination done
[component_container_mt-1] [ERROR] [1724421025.317514297] [isaac_ros_examples.container]: Component constructor threw an exception: [TensorRTNode] Empty engine_file_path_, this needs to be set per the engine
[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'tensor_rt' of type 'nvidia::isaac_ros::dnn_inference::TensorRTNode' in container '/isaac_ros_examples/container': Component constructor threw an exception: [TensorRTNode] Empty engine_file_path_, this needs to be set per the engine
[component_container_mt-1] [INFO] [1724421025.324423496] [isaac_ros_examples.container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_rtdetr/lib/librtdetr_decoder_node.so
[component_container_mt-1] [INFO] [1724421025.329629259] [isaac_ros_examples.container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::rtdetr::RtDetrDecoderNode>
[component_container_mt-1] [INFO] [1724421025.329727942] [isaac_ros_examples.container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::rtdetr::RtDetrDecoderNode>
[component_container_mt-1] [INFO] [1724421025.351558399] [rtdetr_decoder.ManagedNitrosSubscriber]: Starting Managed Nitros Subscriber
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/rtdetr_decoder' in container '/isaac_ros_examples/container'

[component_container_mt-1] 2024-08-23 21:50:35.073 WARN  gxf/std/yaml_file_loader.cpp@1077: Using unregistered parameter 'dummy_rx' in component ''.
[component_container_mt-1] 2024-08-23 21:50:35.074 WARN  gxf/std/yaml_file_loader.cpp@1077: Using unregistered parameter 'dummy_rx' in component ''.
[component_container_mt-1] 2024-08-23 21:50:35.076 WARN  gxf/std/yaml_file_loader.cpp@1077: Using unregistered parameter 'dev_id' in component 'stream'.
[component_container_mt-1] [INFO] [1724421035.079313486] [foundationpose_node]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] 2024-08-23 21:50:35.103 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] 2024-08-23 21:50:35.103 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] 2024-08-23 21:50:35.103 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] 2024-08-23 21:50:35.103 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] 2024-08-23 21:50:35.107 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] [INFO] [1724421035.128228896] [foundationpose_node]: [NitrosNode] Node was started
[component_container_mt-1] 2024-08-23 21:50:35.811 WARN  ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@281: Rebuilding CUDA engine /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.plan (forced by config). Note: this process may take up to several minutes.
[component_container_mt-1] Could not open file /tmp/refine_model.onnx
[component_container_mt-1] Could not open file /tmp/refine_model.onnx
[component_container_mt-1] 2024-08-23 21:50:42.242 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@151: TRT ERROR: ModelImporter.cpp:730: Failed to parse ONNX model from file: /tmp/refine_model.onnx
[component_container_mt-1] 2024-08-23 21:50:42.242 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@472: Failed to parse ONNX file /tmp/refine_model.onnx
[component_container_mt-1] 2024-08-23 21:50:42.621 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@287: Failed to create engine plan for model /tmp/refine_model.onnx.
[component_container_mt-1] 2024-08-23 21:50:42.622 WARN  gxf/std/entity_executor.cpp@495: Failed to start entity [IFHCSCJKZE_refine_inference]
[component_container_mt-1] 2024-08-23 21:50:42.622 WARN  gxf/std/multi_thread_scheduler.cpp@342: Error while executing entity E415 named 'IFHCSCJKZE_refine_inference': GXF_FAILURE
[component_container_mt-1] 2024-08-23 21:50:42.625 ERROR gxf/std/entity_executor.cpp@586: Entity [IFHCSCJKZE_refine_inference] must be in Started, Tick Pending, Ticking or Idle stage before stopping. Current state is StartPending
[component_container_mt-1] 2024-08-23 21:50:43.128 ERROR gxf/std/entity_executor.cpp@210: Entity with eid 296 not found!
[component_container_mt-1] [WARN] [1724421043.128632355] [foundationpose_node]: [NitrosNode] The heartbeat entity (eid=296) was stopped. The graph may have been terminated.
[component_container_mt-1] [INFO] [1724421043.168373716] [foundationpose_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.168430993] [foundationpose_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] 2024-08-23 21:50:43.168 ERROR gxf/core/runtime.cpp@1468: Graph interrupt failed with error: GXF_FAILURE
[component_container_mt-1] [ERROR] [1724421043.168616776] [foundationpose_node]: [NitrosContext] GxfGraphInterrupt Error: GXF_FAILURE
[component_container_mt-1] [INFO] [1724421043.168628744] [foundationpose_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] 2024-08-23 21:50:43.168 ERROR gxf/std/program.cpp@574: wait failed. Deactivating...
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] 2024-08-23 21:50:43.211 WARN  gxf/std/entity_warden.cpp@464: Component of type nvidia::gxf::MultiThreadScheduler, cid 540 failed to deinitialize with code GXF_FAILURE
[component_container_mt-1] 2024-08-23 21:50:43.211 ERROR gxf/core/runtime.cpp@751: Could not deinitialize entity 'IFHCSCJKZE_utils' (E538): GXF_FAILURE
[component_container_mt-1] 2024-08-23 21:50:43.211 ERROR gxf/std/program.cpp@576: Deactivation failed.
[component_container_mt-1] 2024-08-23 21:50:43.211 ERROR gxf/core/runtime.cpp@1476: Graph wait failed with error: GXF_FAILURE
[component_container_mt-1] [ERROR] [1724421043.211704406] [foundationpose_node]: [NitrosContext] GxfGraphWait Error: GXF_FAILURE
[component_container_mt-1] [INFO] [1724421043.211727989] [foundationpose_node]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.255598172] [resize_left_viz]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.255647642] [resize_left_viz]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.256447219] [resize_left_viz]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.256476786] [resize_left_viz]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1724421043.258284538] [resize_left_viz]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.258637160] [resize_left_viz]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.285778335] [resize_mask_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.285822045] [resize_mask_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.287726880] [resize_mask_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.287767454] [resize_mask_node]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1724421043.290581173] [resize_mask_node]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.290948323] [resize_mask_node]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.351694357] [reshape_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.351736435] [reshape_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.352350165] [reshape_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.352375828] [reshape_node]: [NitrosContext] Deinitializing...
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] [INFO] [1724421043.358963315] [reshape_node]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.359194056] [reshape_node]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.372938314] [interleaved_to_planar_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.372984616] [interleaved_to_planar_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.373586347] [interleaved_to_planar_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.373608138] [interleaved_to_planar_node]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1724421043.380180298] [interleaved_to_planar_node]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.380385504] [interleaved_to_planar_node]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.418566748] [resize_left_rt_detr_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.418610330] [resize_left_rt_detr_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.422785167] [resize_left_rt_detr_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.422830509] [resize_left_rt_detr_node]: [NitrosContext] Deinitializing...
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] [INFO] [1724421043.425448493] [resize_left_rt_detr_node]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.425770494] [resize_left_rt_detr_node]: [NitrosNode] Application termination done
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[INFO] [component_container_mt-1]: process has finished cleanly [pid 2072247]

I have already tried the method described in the NVIDIA Developer Forums post at https://forums.developer.nvidia.com/t/isaac-ros-foundationpose-for-pallets/296926/20 and have successfully trained a YOLOv8s model for object detection. I’ve also created my own .obj and .png files, and the mask image output looks good. However, I’m not receiving any pose outputs. I’m wondering if this could be related to the fact that my input image size is 640x320. Could you help me understand if the image size is causing this issue?

I am grateful for any advice or guidance you can provide. I am eager to resolve this issue and continue working with ISAAC FoundationPose.

Thank you in advance for your help.

Best regards,

2

Hi @user29288

Thank you for your post and your tests.

It seems that you are encountering an error due to an incorrect path for the engine. Have you verified that the files are located in the correct folder?

Let me know

Raffaello

3

hi,

I trained a rt-detr model follow the RT-DETR (Realtime Detection Transformer) - Ultralytics YOLO Docs, and covert it using yolo export model=./weights/rt-detr-l.pt format=onnx

and then covert it using
/usr/src/tensorrt/bin/trtexec --onnx=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan admin@ubuntu:/workspaces/isaac_ros-dev$ /usr/src/tensorrt/bin/trtexec --onnx=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan

[08/26/2024-10:25:19] [I] === Model Options ===
[08/26/2024-10:25:19] [I] Format: ONNX
[08/26/2024-10:25:19] [I] Model: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx
[08/26/2024-10:25:19] [I] Output:
[08/26/2024-10:25:19] [I] === Build Options ===
[08/26/2024-10:25:19] [I] Max batch: explicit batch
[08/26/2024-10:25:19] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[08/26/2024-10:25:19] [I] minTiming: 1
[08/26/2024-10:25:19] [I] avgTiming: 8
[08/26/2024-10:25:19] [I] Precision: FP32
[08/26/2024-10:25:19] [I] LayerPrecisions: 
[08/26/2024-10:25:19] [I] Layer Device Types: 
[08/26/2024-10:25:19] [I] Calibration: 
[08/26/2024-10:25:19] [I] Refit: Disabled
[08/26/2024-10:25:19] [I] Version Compatible: Disabled
[08/26/2024-10:25:19] [I] ONNX Native InstanceNorm: Disabled
[08/26/2024-10:25:19] [I] TensorRT runtime: full
[08/26/2024-10:25:19] [I] Lean DLL Path: 
[08/26/2024-10:25:19] [I] Tempfile Controls: { in_memory: allow, temporary: allow }
[08/26/2024-10:25:19] [I] Exclude Lean Runtime: Disabled
[08/26/2024-10:25:19] [I] Sparsity: Disabled
[08/26/2024-10:25:19] [I] Safe mode: Disabled
[08/26/2024-10:25:19] [I] Build DLA standalone loadable: Disabled
[08/26/2024-10:25:19] [I] Allow GPU fallback for DLA: Disabled
[08/26/2024-10:25:19] [I] DirectIO mode: Disabled
[08/26/2024-10:25:19] [I] Restricted mode: Disabled
[08/26/2024-10:25:19] [I] Skip inference: Disabled
[08/26/2024-10:25:19] [I] Save engine: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.plan
[08/26/2024-10:25:19] [I] Load engine: 
[08/26/2024-10:25:19] [I] Profiling verbosity: 0
[08/26/2024-10:25:19] [I] Tactic sources: Using default tactic sources
[08/26/2024-10:25:19] [I] timingCacheMode: local
[08/26/2024-10:25:19] [I] timingCacheFile: 
[08/26/2024-10:25:19] [I] Heuristic: Disabled
[08/26/2024-10:25:19] [I] Preview Features: Use default preview flags.
[08/26/2024-10:25:19] [I] MaxAuxStreams: -1
[08/26/2024-10:25:19] [I] BuilderOptimizationLevel: -1
[08/26/2024-10:25:19] [I] Input(s)s format: fp32:CHW
[08/26/2024-10:25:19] [I] Output(s)s format: fp32:CHW
[08/26/2024-10:25:19] [I] Input build shapes: model
[08/26/2024-10:25:19] [I] Input calibration shapes: model
[08/26/2024-10:25:19] [I] === System Options ===
[08/26/2024-10:25:19] [I] Device: 0
[08/26/2024-10:25:19] [I] DLACore: 
[08/26/2024-10:25:19] [I] Plugins:
[08/26/2024-10:25:19] [I] setPluginsToSerialize:
[08/26/2024-10:25:19] [I] dynamicPlugins:
[08/26/2024-10:25:19] [I] ignoreParsedPluginLibs: 0
[08/26/2024-10:25:19] [I] 
[08/26/2024-10:25:19] [I] === Inference Options ===
[08/26/2024-10:25:19] [I] Batch: Explicit
[08/26/2024-10:25:19] [I] Input inference shapes: model
[08/26/2024-10:25:19] [I] Iterations: 10
[08/26/2024-10:25:19] [I] Duration: 3s (+ 200ms warm up)
[08/26/2024-10:25:19] [I] Sleep time: 0ms
[08/26/2024-10:25:19] [I] Idle time: 0ms
[08/26/2024-10:25:19] [I] Inference Streams: 1
[08/26/2024-10:25:19] [I] ExposeDMA: Disabled
[08/26/2024-10:25:19] [I] Data transfers: Enabled
[08/26/2024-10:25:19] [I] Spin-wait: Disabled
[08/26/2024-10:25:19] [I] Multithreading: Disabled
[08/26/2024-10:25:19] [I] CUDA Graph: Disabled
[08/26/2024-10:25:19] [I] Separate profiling: Disabled
[08/26/2024-10:25:19] [I] Time Deserialize: Disabled
[08/26/2024-10:25:19] [I] Time Refit: Disabled
[08/26/2024-10:25:19] [I] NVTX verbosity: 0
[08/26/2024-10:25:19] [I] Persistent Cache Ratio: 0
[08/26/2024-10:25:19] [I] Inputs:
[08/26/2024-10:25:19] [I] === Reporting Options ===
[08/26/2024-10:25:19] [I] Verbose: Disabled
[08/26/2024-10:25:19] [I] Averages: 10 inferences
[08/26/2024-10:25:19] [I] Percentiles: 90,95,99
[08/26/2024-10:25:19] [I] Dump refittable layers:Disabled
[08/26/2024-10:25:19] [I] Dump output: Disabled
[08/26/2024-10:25:19] [I] Profile: Disabled
[08/26/2024-10:25:19] [I] Export timing to JSON file: 
[08/26/2024-10:25:19] [I] Export output to JSON file: 
[08/26/2024-10:25:19] [I] Export profile to JSON file: 
[08/26/2024-10:25:19] [I] 
[08/26/2024-10:25:20] [I] === Device Information ===
[08/26/2024-10:25:20] [I] Selected Device: Orin
[08/26/2024-10:25:20] [I] Compute Capability: 8.7
[08/26/2024-10:25:20] [I] SMs: 16
[08/26/2024-10:25:20] [I] Device Global Memory: 62841 MiB
[08/26/2024-10:25:20] [I] Shared Memory per SM: 164 KiB
[08/26/2024-10:25:20] [I] Memory Bus Width: 256 bits (ECC disabled)
[08/26/2024-10:25:20] [I] Application Compute Clock Rate: 1.3 GHz
[08/26/2024-10:25:20] [I] Application Memory Clock Rate: 0.816 GHz
[08/26/2024-10:25:20] [I] 
[08/26/2024-10:25:20] [I] Note: The application clock rates do not reflect the actual clock rates that the GPU is currently running at.
[08/26/2024-10:25:20] [I] 
[08/26/2024-10:25:20] [I] TensorRT version: 8.6.2
[08/26/2024-10:25:20] [I] Loading standard plugins
[08/26/2024-10:25:20] [I] [TRT] [MemUsageChange] Init CUDA: CPU +2, GPU +0, now: CPU 33, GPU 11932 (MiB)
[08/26/2024-10:25:26] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +1154, GPU +1115, now: CPU 1223, GPU 13067 (MiB)
[08/26/2024-10:25:26] [I] Start parsing network model.
Could not open file /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx
Could not open file /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx
[08/26/2024-10:25:26] [E] [TRT] ModelImporter.cpp:730: Failed to parse ONNX model from file: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx
[08/26/2024-10:25:26] [E] Failed to parse onnx file
[08/26/2024-10:25:26] [I] Finished parsing network model. Parse time: 0.00179726
[08/26/2024-10:25:26] [E] Parsing model failed
[08/26/2024-10:25:26] [E] Failed to create engine from model or file.
[08/26/2024-10:25:26] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8602] # /usr/src/tensorrt/bin/trtexec --onnx=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx --saveEngine=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.plan
admin@ubuntu:/workspaces/isaac_ros-dev$ /usr/src/tensorrt/bin/trtexec --onnx=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan
&&&& RUNNING TensorRT.trtexec [TensorRT v8602] # /usr/src/tensorrt/bin/trtexec --onnx=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan
[08/26/2024-10:27:27] [I] === Model Options ===
[08/26/2024-10:27:27] [I] Format: ONNX
[08/26/2024-10:27:27] [I] Model: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx
[08/26/2024-10:27:27] [I] Output:
[08/26/2024-10:27:27] [I] === Build Options ===
[08/26/2024-10:27:27] [I] Max batch: explicit batch
[08/26/2024-10:27:27] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[08/26/2024-10:27:27] [I] minTiming: 1
[08/26/2024-10:27:27] [I] avgTiming: 8
[08/26/2024-10:27:27] [I] Precision: FP32
[08/26/2024-10:27:27] [I] LayerPrecisions: 
[08/26/2024-10:27:27] [I] Layer Device Types: 
[08/26/2024-10:27:27] [I] Calibration: 
[08/26/2024-10:27:27] [I] Refit: Disabled
[08/26/2024-10:27:27] [I] Version Compatible: Disabled
[08/26/2024-10:27:27] [I] ONNX Native InstanceNorm: Disabled
[08/26/2024-10:27:27] [I] TensorRT runtime: full
[08/26/2024-10:27:27] [I] Lean DLL Path: 
[08/26/2024-10:27:27] [I] Tempfile Controls: { in_memory: allow, temporary: allow }
[08/26/2024-10:27:27] [I] Exclude Lean Runtime: Disabled
[08/26/2024-10:27:27] [I] Sparsity: Disabled
[08/26/2024-10:27:27] [I] Safe mode: Disabled
[08/26/2024-10:27:27] [I] Build DLA standalone loadable: Disabled
[08/26/2024-10:27:27] [I] Allow GPU fallback for DLA: Disabled
[08/26/2024-10:27:27] [I] DirectIO mode: Disabled
[08/26/2024-10:27:27] [I] Restricted mode: Disabled
[08/26/2024-10:27:27] [I] Skip inference: Disabled
[08/26/2024-10:27:27] [I] Save engine: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan
[08/26/2024-10:27:27] [I] Load engine: 
[08/26/2024-10:27:27] [I] Profiling verbosity: 0
[08/26/2024-10:27:27] [I] Tactic sources: Using default tactic sources
[08/26/2024-10:27:27] [I] timingCacheMode: local
[08/26/2024-10:27:27] [I] timingCacheFile: 
[08/26/2024-10:27:27] [I] Heuristic: Disabled
[08/26/2024-10:27:27] [I] Preview Features: Use default preview flags.
[08/26/2024-10:27:27] [I] MaxAuxStreams: -1
[08/26/2024-10:27:27] [I] BuilderOptimizationLevel: -1
[08/26/2024-10:27:27] [I] Input(s)s format: fp32:CHW
[08/26/2024-10:27:27] [I] Output(s)s format: fp32:CHW
[08/26/2024-10:27:27] [I] Input build shapes: model
[08/26/2024-10:27:27] [I] Input calibration shapes: model
[08/26/2024-10:27:27] [I] === System Options ===
[08/26/2024-10:27:27] [I] Device: 0
[08/26/2024-10:27:27] [I] DLACore: 
[08/26/2024-10:27:27] [I] Plugins:
[08/26/2024-10:27:27] [I] setPluginsToSerialize:
[08/26/2024-10:27:27] [I] dynamicPlugins:
[08/26/2024-10:27:27] [I] ignoreParsedPluginLibs: 0
[08/26/2024-10:27:27] [I] 
[08/26/2024-10:27:27] [I] === Inference Options ===
[08/26/2024-10:27:27] [I] Batch: Explicit
[08/26/2024-10:27:27] [I] Input inference shapes: model
[08/26/2024-10:27:27] [I] Iterations: 10
[08/26/2024-10:27:27] [I] Duration: 3s (+ 200ms warm up)
[08/26/2024-10:27:27] [I] Sleep time: 0ms
[08/26/2024-10:27:27] [I] Idle time: 0ms
[08/26/2024-10:27:27] [I] Inference Streams: 1
[08/26/2024-10:27:27] [I] ExposeDMA: Disabled
[08/26/2024-10:27:27] [I] Data transfers: Enabled
[08/26/2024-10:27:27] [I] Spin-wait: Disabled
[08/26/2024-10:27:27] [I] Multithreading: Disabled
[08/26/2024-10:27:27] [I] CUDA Graph: Disabled
[08/26/2024-10:27:27] [I] Separate profiling: Disabled
[08/26/2024-10:27:27] [I] Time Deserialize: Disabled
[08/26/2024-10:27:27] [I] Time Refit: Disabled
[08/26/2024-10:27:27] [I] NVTX verbosity: 0
[08/26/2024-10:27:27] [I] Persistent Cache Ratio: 0
[08/26/2024-10:27:27] [I] Inputs:
[08/26/2024-10:27:27] [I] === Reporting Options ===
[08/26/2024-10:27:27] [I] Verbose: Disabled
[08/26/2024-10:27:27] [I] Averages: 10 inferences
[08/26/2024-10:27:27] [I] Percentiles: 90,95,99
[08/26/2024-10:27:27] [I] Dump refittable layers:Disabled
[08/26/2024-10:27:27] [I] Dump output: Disabled
[08/26/2024-10:27:27] [I] Profile: Disabled
[08/26/2024-10:27:27] [I] Export timing to JSON file: 
[08/26/2024-10:27:27] [I] Export output to JSON file: 
[08/26/2024-10:27:27] [I] Export profile to JSON file: 
[08/26/2024-10:27:27] [I] 
[08/26/2024-10:27:27] [I] === Device Information ===
[08/26/2024-10:27:27] [I] Selected Device: Orin
[08/26/2024-10:27:27] [I] Compute Capability: 8.7
[08/26/2024-10:27:27] [I] SMs: 16
[08/26/2024-10:27:27] [I] Device Global Memory: 62841 MiB
[08/26/2024-10:27:27] [I] Shared Memory per SM: 164 KiB
[08/26/2024-10:27:27] [I] Memory Bus Width: 256 bits (ECC disabled)
[08/26/2024-10:27:27] [I] Application Compute Clock Rate: 1.3 GHz
[08/26/2024-10:27:27] [I] Application Memory Clock Rate: 0.816 GHz
[08/26/2024-10:27:27] [I] 
[08/26/2024-10:27:27] [I] Note: The application clock rates do not reflect the actual clock rates that the GPU is currently running at.
[08/26/2024-10:27:27] [I] 
[08/26/2024-10:27:27] [I] TensorRT version: 8.6.2
[08/26/2024-10:27:27] [I] Loading standard plugins
[08/26/2024-10:27:27] [I] [TRT] [MemUsageChange] Init CUDA: CPU +2, GPU +0, now: CPU 33, GPU 11906 (MiB)
[08/26/2024-10:27:33] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +1154, GPU +1112, now: CPU 1223, GPU 13030 (MiB)
[08/26/2024-10:27:33] [I] Start parsing network model.
Could not open file /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx
Could not open file /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx
[08/26/2024-10:27:33] [E] [TRT] ModelImporter.cpp:730: Failed to parse ONNX model from file: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx
[08/26/2024-10:27:33] [E] Failed to parse onnx file
[08/26/2024-10:27:33] [I] Finished parsing network model. Parse time: 0.00180772
[08/26/2024-10:27:33] [E] Parsing model failed
[08/26/2024-10:27:33] [E] Failed to create engine from model or file.
[08/26/2024-10:27:33] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8602] # /usr/src/tensorrt/bin/trtexec --onnx=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan

I have uploaded my custom rt-detr model for you to test on your side, thank you.

rtdetr-l.zip (64.8 MB)

4

Hi @user29288

Have you fixed your issue?

Best,
Raffaello

5

Hello, Raffaello, I also need your help about the custom rt_dert model for isaac_ros. I trained a rt_dert model and transfer it to onnx by using “yolo export model=best.pt format=onnx”, and then, using:
/usr/src/tensorrt/bin/trtexec --onnx=${ISAAC_ROS_WS}/isaac_ros_assets/models/synthetica_detr/best.onnx --saveEngine=${ISAAC_ROS_WS}/isaac_ros_assets/models/synthetica_detr/best.plan
to change it to plan file.
However, When I run as:
ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_mono_rect,rtdetr engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/synthetica_detr/best.plan
I have:
[component_container_mt-1] 2025-12-11 20:40:59.292 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@330: Numbers of CUDA bindings mismatch: configured for 5 vs model requires 2. Please check TensorRTInference codelet configuration.

I seems that the input and the output have the different sizes for the bindings.
Do you have some suggestion about this error? How to change the model satisfy the requirement of the tensor_rt?

6

Fine, I figure it out. There is something that need to do with the custom model to adapt to the struction, data format, namely to re-construct the output and input of the custom model.
I use rt_detr from the original author in github to train the model, and transfer it to the onnx type by using the following code:

#!/usr/bin/env python3

# export_rtdetr_correct_format.py

import torch

import torch.nn as nn

from ultralytics import RTDETR

import numpy as np

import cv2

class RtDetrCorrectFormat(nn.Module):

“”“修正格式：将中心点+宽高转换为角点坐标”“”

def _init_(self, model_path):

super()._init_()

rtdetr = RTDETR(model_path)

self.model = rtdetr.model

self.num_classes = rtdetr.model.nc

print(f"类别数: {self.num_classes}")

def forward(self, images, orig_target_sizes):

“”"

    关键修正：模型输出是\[cx, cy, w, h\]，需要转换为\[x1, y1, x2, y2\]

    """

orig_target_sizes = orig_target_sizes.to(torch.int64)

# 1. 获取模型输出

outputs = self.model(images)

if isinstance(outputs, tuple):

outputs = outputs[0]

batch_size = outputs.shape[0]

# 2. 提取boxes - 假设格式是[cx, cy, w, h]

boxes_raw = outputs[…, :4] # [batch, 300, 4]

# 3. DEBUG: 打印原始boxes格式

debug = True

if debug and batch_size == 1:

print(f"DEBUG - 原始boxes[0, 0]: {boxes_raw[0, 0].tolist()}")

# 手动计算验证

cx, cy, w, h = boxes_raw[0, 0]

x1 = cx - w/2

y1 = cy - h/2

x2 = cx + w/2

y2 = cy + h/2

print(f"DEBUG - 转换后: [{x1:.1f}, {y1:.1f}, {x2:.1f}, {y2:.1f}]")

# 4. 将[cx, cy, w, h]转换为[x1, y1, x2, y2]

# boxes_raw形状: [batch, 300, 4] 其中4=[cx, cy, w, h]

cx = boxes_raw[…, 0:1] # 保持维度 [batch, 300, 1]

cy = boxes_raw[…, 1:2]

w = boxes_raw[…, 2:3]

h = boxes_raw[…, 3:4]

x1 = cx - w/2

y1 = cy - h/2

x2 = cx + w/2

y2 = cy + h/2

# 合并为[x1, y1, x2, y2]

boxes_corners = torch.cat([x1, y1, x2, y2], dim=-1)

# 5. 坐标缩放到像素坐标

img_h = orig_target_sizes[:, 0:1].unsqueeze(1).float()

img_w = orig_target_sizes[:, 1:2].unsqueeze(1).float()

scale_factor = torch.cat([img_w, img_h, img_w, img_h], dim=-1)

boxes_pixel = boxes_corners * scale_factor

# 6. 限制坐标范围

boxes_pixel = torch.clamp(boxes_pixel, min=0)

max_coords = torch.cat([img_w, img_h, img_w, img_h], dim=-1)

boxes_pixel = torch.min(boxes_pixel, max_coords)

# 7. 提取scores和labels

scores_all = outputs[…, 4:]

scores, labels = torch.max(scores_all, dim=-1)

# 8. 确保orig_target_sizes不被优化

dummy = (img_w.mean() + img_h.mean()) * 0.0

# 9. 返回正确格式

return (labels.to(torch.int64) + dummy.to(torch.int64),

boxes_pixel.to(torch.float32) + dummy,

scores.to(torch.float32) + dummy)

def test_with_visualization():

“”“用可视化验证转换”“”

print(“\n” + “=”*60)

print(“可视化验证格式转换…”)

# 加载图片

img = cv2.imread(“image2.png”)

original_h, original_w = img.shape[:2]

# 预处理

img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

img_resized = cv2.resize(img_rgb, (640, 640))

img_normalized = img_resized.astype(np.float32) / 255.0

img_tensor = torch.from_numpy(img_normalized).permute(2, 0, 1).unsqueeze(0)

# 创建模型

model = RtDetrCorrectFormat(“best.pt”)

model.eval()

# 推理

orig_target_sizes = torch.tensor([[original_h, original_w]], dtype=torch.int64)

with torch.no_grad():

labels, boxes, scores = model(img_tensor, orig_target_sizes)

# 找到最佳检测

best_idx = torch.argmax(scores[0]).item()

best_box = boxes[0, best_idx].tolist()

best_score = scores[0, best_idx].item()

print(f"\n转换后边界框: [{best_box[0]:.1f}, {best_box[1]:.1f}, {best_box[2]:.1f}, {best_box[3]:.1f}]")

width = best_box[2] - best_box[0]

height = best_box[3] - best_box[1]

print(f"宽高: {width:.1f}x{height:.1f}")

# 可视化

vis_img = img.copy()

x1, y1, x2, y2 = map(int, best_box)

# 绘制导出模型的检测框（绿色）

cv2.rectangle(vis_img, (x1, y1), (x2, y2), (0, 255, 0), 3)

label = f"Export: {width:.0f}x{height:.0f}"

cv2.putText(vis_img, label, (x1, y1-10),

cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2)

# 绘制离线模型的检测框（红色）用于对比

offline_box = [95.4, 11.7, 540.1, 309.5]

ox1, oy1, ox2, oy2 = map(int, offline_box)

cv2.rectangle(vis_img, (ox1, oy1), (ox2, oy2), (0, 0, 255), 2)

offline_label = f"Offline: 445x298"

cv2.putText(vis_img, offline_label, (ox1, oy1-40),

cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)

# 显示

cv2.imshow(‘Format Correction Test (Green=Export, Red=Offline)’, vis_img)

print(“\n绿色框应该与红色框完全重合！”)

cv2.waitKey(0)

cv2.destroyAllWindows()

# 计算误差

error_x1 = abs(x1 - ox1)

error_y1 = abs(y1 - oy1)

error_x2 = abs(x2 - ox2)

error_y2 = abs(y2 - oy2)

print(f"\n坐标误差:")

print(f" x1误差: {error_x1} 像素")

print(f" y1误差: {error_y1} 像素")

print(f" x2误差: {error_x2} 像素")

print(f" y2误差: {error_y2} 像素")

if max(error_x1, error_y1, error_x2, error_y2) < 5:

print(“✓ 格式转换成功！”)

return True

else:

print(“⚠ 格式转换仍有误差”)

return False

def export_model():

“”“导出修正后的模型”“”

print(“导出RT-DETR（修正坐标格式）”)

print(“=”*60)

# 先验证转换

if not test_with_visualization():

print(“\n⚠ 格式转换验证失败，请检查模型输出格式”)

return

print(“\n” + “=”*60)

print(“导出ONNX模型…”)

# 创建模型

model = RtDetrCorrectFormat(“best.pt”)

model.eval()

# 准备导出输入

images = torch.randn(1, 3, 640, 640, dtype=torch.float32)

orig_target_sizes = torch.tensor([[480, 640]], dtype=torch.int64)

# 导出

torch.onnx.export(

model,

    (images, orig_target_sizes),

“best3.onnx”,

opset_version=16,

input_names=[‘images’, ‘orig_target_sizes’],

output_names=[‘labels’, ‘boxes’, ‘scores’],

dynamic_axes={

‘images’: {0: ‘batch_size’},

‘orig_target_sizes’: {0: ‘batch_size’},

‘labels’: {0: ‘batch_size’},

‘boxes’: {0: ‘batch_size’},

‘scores’: {0: ‘batch_size’}

    }

)

print(“✓ ONNX导出完成: best3.onnx”)

# 验证

verify_export()

def verify_export():

“”“验证导出结果”“”

import onnxruntime as ort

print(“\n验证导出模型…”)

session = ort.InferenceSession(“best3.onnx”)

# 用测试图片验证

img = cv2.imread(“image2.png”)

original_h, original_w = img.shape[:2]

img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

img_resized = cv2.resize(img_rgb, (640, 640))

img_normalized = img_resized.astype(np.float32) / 255.0

img_input = img_normalized.transpose(2, 0, 1)[np.newaxis, …]

orig_target_sizes_np = np.array([[original_h, original_w]], dtype=np.int64)

outputs = session.run(None, {

‘images’: img_input,

‘orig_target_sizes’: orig_target_sizes_np

})

labels, boxes, scores = outputs

best_idx = np.argmax(scores[0])

best_box = boxes[0, best_idx]

print(f"\nONNX模型输出:")

print(f" 最佳检测框: [{best_box[0]:.1f}, {best_box[1]:.1f}, {best_box[2]:.1f}, {best_box[3]:.1f}]")

width = best_box[2] - best_box[0]

height = best_box[3] - best_box[1]

print(f" 宽高: {width:.1f}x{height:.1f}")

# 检查是否匹配离线结果

expected_width = 444.7

expected_height = 297.8

width_match = abs(width - expected_width) < 2.0

height_match = abs(height - expected_height) < 2.0

if width_match and height_match:

print(f"✓ 完美匹配离线结果！")

else:

print(f"⚠ 仍有差异: 期望{expected_width:.1f}x{expected_height:.1f}, 实际{width:.1f}x{height:.1f}")

if _name_ == “_main_”:

print(“RT-DETR坐标格式修正工具”)

print(“将[cx,cy,w,h]转换为[x1,y1,x2,y2]”)

print(“=”*60)

export_model()

print(“\n” + “=”*60)

print(“转换TensorRT命令:”)

print(“”"

/usr/src/tensorrt/bin/trtexec \\

–onnx=best3.onnx \\

–saveEngine=best3.plan \\

–explicitBatch \\

–minShapes=images:1x3x640x640,orig_target_sizes:1x2 \\

–optShapes=images:1x3x640x640,orig_target_sizes:1x2 \\

–maxShapes=images:4x3x640x640,orig_target_sizes:4x2 \\

–workspace=2048 \\

–fp16

""")

It worked after the onnx model converted to the plan model. BTW, the ImageNormalizeNode should also be added to the launch file to normalize the image.

The key point is to re-contruct the trained model to satisfy the pipeline in isaac_ros_rtdetr.