Official Baseline Implementation for Track 5
Based on Pi3DET -- "Perspective-Invariant 3D Object Detection"
(https://github.com/pi3det/toolkit)
🏆 Prize Pool: $2,000 USD for Track 5 Winners
- [2025-07-04]: We have fixed the bug of 'mot_3d'
- [2025-06-12]: The RoboSense Challenge is online. Track 5: Cross-Platform 3D Object Detection focuses on the development of robust 3D object detectors that can seamlessly adapt across different robot platforms, including vehicles, drones, and quadrupeds.
Participants are expected to develop new adaptation algorithms that can effectively adapt 3D perception tasks, specifically object detection, across three robot platforms that use different sensor configurations and movement dynamics. The models are expected to be trained using vehicle data, and achieve promising performance on drone and quadruped platforms.
- Venue: IROS 2025, Hangzhou (Oct 19-25, 2025)
- Registration: Google Form (Open until Aug 15)
- Contact: [email protected]
| Prize | Award |
|---|---|
| 🥇 1st Place | $1000 + Certificate |
| 🥈 2nd Place | $600 + Certificate |
| 🥉 3rd Place | $400 + Certificate |
| 🌟 Innovation Award | Cash Award + Certificate |
| Participation | Certificate |
The Cross-Platform Track is structured into two consecutive phases:
- Duration: 15 June 2025 – 15 August 2025
- Setup:
- Source domain: Vehicle LiDAR scans with 3D bounding-box annotations
- Target domain: Unlabeled Drone LiDAR scans
- Ranking metric: [email protected] (R40) for the Car class evaluated on Drone data
- Duration: 15 August 2025 – 15 October 2025
- Setup:
- Source domain: Vehicle LiDAR scans with annotations
- Target domains: Unlabeled Drone and Quadruped LiDAR scans
- Ranking metric: Weighted score combining:
- [email protected] (R40) for the Car class
- [email protected] (R40) for the Pedestrian class
(Scores computed across both Drone and Quadruped platforms.)
More ranking details are shown at Ranking.
This track is developed on top of the popular 3D detection codebase OpenPCDet. To avoid build failures, make sure your CUDA version matches your PyTorch installation before proceeding. All the installation and testing process had beed tested with PyTorch 2.1.0-cu118 on Ubuntu 22.04.
- Clone the repository
git clone https://github.com/robosense2025/track5.git
- Enter the project directory
cd track5 - Install dependencies
pip install -r requirements.txt
- Build and install the
pcdetpackagepython setup.py develop
- Verify installation
pip list | grep pcdet
If you run into any installation issues, please open an issue on the GitHub repo or contact us via the WeChat group. Happy coding!
The Track 5 dataset follows the KITTI format. Each sample consists of:
- A front-view RGB image
- A LiDAR point cloud covering the camera’s field of view
- Calibration parameters
- 3D bounding-box annotations (for training)
Calibration and annotations are packaged together in
.pklfiles.
We use the same training set (vehicle platform) for both phases, but different validation sets. The full dataset is hosted on Hugging Face:
robosense/track5-cross-platform-3d-object-detection
- Download the dataset
# export your huggingface token: hf_xxx export HUGGINGFACE_TOKEN=${TOKEN} python tools/load_dataset.py $USER_DEFINE_OUTPUT_PATH
- Link data into the project
# Create target directory mkdir -p data/pi3det # Link the training split ln -s $USER_DEFINE_OUTPUT_PATH/track5-cross-platform-3d-object-detection/phase12_vehicle_training/training \ data/pi3det/training # Link the validation split for Phase N (Drone or Quadruped) ln -s $USER_DEFINE_OUTPUT_PATH/track5-cross-platform-3d-object-detection/phase{$N}_{$PLATFORM}_validation/validation \ data/pi3det/validation # Link the .pkl info files ln -s $USER_DEFINE_OUTPUT_PATH/track5-cross-platform-3d-object-detection/phase12_vehicle_training/training/pi3det_infos_train.pkl \ data/pi3det/pi3det_infos_train.pkl ln -s $USER_DEFINE_OUTPUT_PATH/track5-cross-platform-3d-object-detection/phase{$N}_{$PLATFORM}_validation/pi3det_infos_val.pkl \ data/pi3det/pi3det_infos_val.pkl
- Verify your directory structure
After linking, yourdata/folder should look like this:data/ └── pi3det/ ├── training/ │ ├── image/ │ │ ├── 0000000.jpg │ │ └── 0000001.jpg │ └── point_cloud/ │ ├── 0000000.bin │ └── 0000001.bin ├── validation/ │ ├── image/ │ │ ├── 0000000.jpg │ │ └── 0000001.jpg │ └── point_cloud/ │ ├── 0000000.bin │ └── 0000001.bin ├── pi3det_infos_train.pkl └── pi3det_infos_val.pkl
The purpose of Cross Platform is like an Unsupervised Domain Adaptation (UDA) task is to learn a generalized model or backbone
Here, we take Phase1: Vehicle-to-Drone adaptation as an example. We use PVRCNN as our base detector.
OpenPCDusetoolsas project workspace
cd tools- Train FEAT=3 (X,Y,Z) using multiple GPUs
bash scripts/dist_train.sh ${NUM_GPUs} \
--cfg_file ./cfgs/DA/phase1_vehicle_drone/source_only/pvrcnn_source.yaml- Train FEAT=3 (X,Y,Z) using single GPU
python train.py --cfg_file ./cfgs/DA/phase1_vehicle_drone/source_only/pvrcnn_source.yamlHere, we take Phase1: Vehicle-to-Drone adaptation as an example. We use ST3D as our baseline method.
- Train FEAT=3 (X,Y,Z) using multiple GPUs
sh scripts/UDA/dist_train_uda.sh ${NUM_GPUs} \
--cfg_file ./cfgs/DA/phase{$N}_vehicle_{$PLATFORM}/st3d/pvrcnn_st3d.yaml \
--pretrained_model ${PRETRAINED_MODEL}- Train FEAT=3 (X,Y,Z) using single GPU
python train_uda.py \
--cfg_file ./cfgs/DA/phase{$N}_vehicle_{$PLATFORM}/st3d/pvrcnn_st3d.yaml \
--pretrained_model ${PRETRAINED_MODEL}$PRETRAINED_MODEL is pretrained from the source platform in Source training.
The validation set for this track does not include annotation files. All results must be submitted and evaluated through the competition submission website. We have lifted any rate limits on validation submissions so you can evaluate as often as needed.
- Test with a ckpt file:
python test.py \
--cfg_file ${CONFIG_FILE} \
--batch_size ${BATCH_SIZE} \
--ckpt ${CKPT}- To test all the saved checkpoints of a specific training setting and draw the performance curve on the Tensorboard, add the
--eval_allargument:
python test.py \
--cfg_file ${CONFIG_FILE} \
--batch_size ${BATCH_SIZE} \
--eval_all- To test with multiple GPUs:
sh scripts/dist_test.sh ${NUM_GPUs} \
--cfg_file ${CONFIG_FILE} \
--batch_size ${BATCH_SIZE} \
--ckpt ${CKPT}- To test all checkpoints with multiple GPUs
sh scripts/dist_test.sh ${NUM_GPUs} \
--cfg_file ${CONFIG_FILE} \
--batch_size ${BATCH_SIZE} \
--eval_allOnce testing completes, you will find a result.pkl file in your output directory. Please compress the file directly to result.zip and this file is your submission payload for the leaderboard.
We report the cross-platform adaptation results, including phase 1 and phase 2.
- All LiDAR-based models are trained with 2 NVIDIA T8 GPUs and are available for download.
- The platform adaptation time is measured with 2 NVIDIA T8 GPUs and PyTorch 2.1.0-cu118.
| Adaptation | Car AP0.7@R40 | Car AP0.5@R40 | download | |
|---|---|---|---|---|
| PV-RCNN | Source-only | 34.60 / 16.31 | 40.67 / 33.70 | checkpoint |
| PV-RCNN | ST3D | 47.81 / 26.03 | 53.40 / 46.64 | checkpoint |
| PV-RCNN | ST3D++ | 45.96 / 25.37 | 52.65 / 45.07 | checkpoint |
| Adaptation | Car AP0.5@R40 | Pedestrian AP0.5@R40 | download | |
|---|---|---|---|---|
| PV-RCNN | Source-only | 26.86 / 22.24 | 42.29 / 37.54 | checkpoint |
| PV-RCNN | ST3D | 34.60 / 28.97 | 48.68 / 43.51 | checkpoint |
| PV-RCNN | ST3D++ | 32.76 / 28.53 | 46.99 / 41.49 | checkpoint |
Beyond the provided baseline, participants are encouraged to explore alternative strategies to further boost cross-platform performance:
- Treat the cross-platform challenge as a domain adaptation problem by improving pseudo-label quality and fine-tuning on target-platform data.
- Design novel data augmentation techniques to bridge geometric and feature discrepancies across platforms.
- Adopt geometry-agnostic 3D detectors, such as point-based architectures, that are less sensitive to platform-specific point-cloud characteristics.
We provide a lightweight UI to help you interactively explore the dataset and your model's predictions. Before you begin, make sure you have downloaded and linked the data as described in Getting Started.
Run the following command in your project root:
python ./vis_tools/active_window.py
- Select Split
- Use the dropdown menu at the top to choose training or validation.
- Click Load Dataset
- Inspect Views
- Left panel: front-view RGB image
- Right panel: LiDAR point cloud (FOV region)
- Ground-truth 3D boxes are overlaid on the LiDAR view.
- After testing, locate the
result.pklfile in your output directory. - Click Load Anno and select
result.pkl. - Predicted 3D bounding boxes will appear on point-cloud panels.
- Registration: Google Form
- Phase 1 Deadline: August 15th
- Phase 2 Deadline: September 15th
- Awards Announcement: IROS 2025
- Challenge Website: robosense2025.github.io
- Track Details: Track 5 Page
- Related Paper:
- Email: [email protected]
- Official Website: https://robosense2025.github.io
- Issues: Please use GitHub Issues for technical questions
If you use the code and dataset in your research, please cite:
@article{liang2025perspective,
title={Perspective-Invariant 3D Object Detection},
author={Liang, Ao and Kong, Lingdong and Lu, Dongyue and Liu, Youquan and Fang, Jian and Zhao, Huaici and Ooi, Wei Tsang},
journal={arXiv preprint arXiv:2507.17665},
year={2025}
}
🤖 Ready to sense the world robustly? Register now and compete for $2,000!
📝 Register Here | 🌐 Challenge Website | 📧 Contact Us
Made with ❤️ by the RoboSense 2025 Team