Night-Voyager: Consistent and Efficient Nocturnal Vision-Aided State Estimation in Object Maps

News

• 2025-03-02: The code and the collected nighttime dataset are released.
• 2025-02-18: The paper is accepted by IEEE Transactions on Robotics (T-RO), 2025.

1. Introduction

Night-Voyager is a consistent and efficient framework that harnesses rare object-level information from streetlight maps to fundamentally resolve the insufficiency and inconsistency bottlenecks inherent in nocturnal visual tasks, achieving robust and versatile nocturnal vision-aided state estimation in object maps.

2. Dataset

The collected nighttime dataset (Night-Voyager Dataset) is also available online. Each scenario in the dataset comprises two sets: one for constructing the streetlight map and the other for evaluating the algorithms. The streetlight detections are recorded in a separate folder and the constructed streetlight maps are placed in another folder. More details of the Night-Voyager Dataset can be found here. Additionally, we also leverage the nighttime sequences of the public MCD Dataset for evaluation.

3. Prerequisites

3.1. Ubuntu and ROS

Ubuntu 18.04.

ROS Melodic, please follow ROS Installation.

3.2. PCL, OpenCV, and Eigen

PCL 1.8, please follow PCL Installation.

OpenCV 3.2.0, please follow OpenCV Installation.

Eigen 3.3.4, please follow Eigen Installation.

4. Build

Clone the repository and catkin_make:

mkdir -p ws_Night_Voyager/src
cd ~/ws_Night_Voyager/src
git clone https://github.com/IMRL/Night-Voyager.git
cd ../
# Maybe need to compile twice for custom ROS messages
catkin_make && catkin_make
source ~/ws_Night_Voyager/devel/setup.bash

5. Run

Download our collected rosbag files via Baidu NetDisk (Night-Voyager Dataset). Since the ROS bag playback node is already included in the launch files, please update the file path ("data_path") in the launch files (located in the launch subdirectory) to match the downloaded ROS bag files. Additionally, by modifying the "scene_name" parameter, you can test Night-Voyager with the corresponding sequence.

roslaunch night_voyager night_voyager.launch

6. Night-Voyager Dataset

The characters of each sequence are summarized in the following table:

Sequence	Distance	Scene	Distribution	Preview	Map
Scene_01	724 m	Avenue & Garden	Sparse
Scene_02	613 m	Lane	Dense
Scene_03	635 m	Lane & Pavement	Sparse
Scene_04	305 m	Garden	Sparse
Scene_05	777 m	Seaside Road	Dense
Scene_06	1071 m	Seaside Road	Dense
Scene_07	892 m	Avenue & Garden	Mixed
Scene_08	719 m	Alley & Campus	Mixed
Scene_09	842 m	Campus	Mixed
Scene_10	601 m	Bridge & Avenue	Mixed

7. Citation

If you find the works beneficial to your research, you may consider citing:

@inproceedings{gao2024night,
  title={Night-Rider: Nocturnal Vision-aided Localization in Streetlight Maps Using Invariant Extended Kalman Filtering},
  author={Gao, Tianxiao and Zhao, Mingle and Xu, Chengzhong and Kong, Hui},
  booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={10238--10244},
  year={2024},
  organization={IEEE}
}

@article{gao2025night,
  title={Night-Voyager: Consistent and Efficient Nocturnal Vision-Aided State Estimation in Object Maps},
  author={Gao, Tianxiao and Zhao, Mingle and Xu, Chengzhong and Kong, Hui},
  journal={IEEE Transactions on Robotics},
  year={2025},
  publisher={IEEE}
}

8. Acknowledgement

The code references the implementation of the P3P solver and OpenVINS. We thank the authors for their fantastic works!

9. Contact

• For technical issues and support, please contact Tianxiao Gao at [email protected] or Mingle Zhao at [email protected].
• For commercial use, please contact Prof. Hui Kong at [email protected].