Projection-Based NBV planning Framework

Overview

• We propose a novel projection-based Next-Best-View (NBV) planning framework that aims to alleviate the high computational burden associated with existing algorithms due to the use of ray-casting in the process of selecting the best viewpoint.

Experiment platform

(a) Real-world platform

(b) Simulation platform (Similar to real-world platforms using a robotic arm, a turntable and a struct light 3D camera.)

(c) Simple environment just using a RealSense

Experiment Result

[video]

Dependencies

• Install Ros
  • http://wiki.ros.org/noetic/Installation/Ubuntu (use full desktop install, include opencv, pcl)
• Install Dependencies
  • apt install libnlopt-dev libnlopt-cxx-dev ros-noetic-geometric-shapes libjsoncpp-dev libcgal-dev ttf-mscorefonts-installer
  • apt-get install ros-noetic-soem ros-noetic-gazebo-ros-control ros-noetic-ros-control ros-noetic-ros-controllers ros-noetic-moveit-* ros-noetic-trac-ik ros-noetic-octovis ros-noetic-behaviortree-cpp-v3 psmisc (add for robotic arm control)
• Compile Dependencies
  • pb_nbv/3rdparty/lemon-1.3.1
  • pb_nbv/3rdparty/gflag-2.2.2 (-DBUILD_SHARED_LIBS="on")
  • pb_nbv/3rdparty/gflag-0.6.0
  • pb_nbv/3rdparty/CSerialPort-4.3.1 (for real-world experiment)
• Python Dependencies
  • pip install networkx==3.1
  • pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
  • pip install open3d trimehs pycollada
• Tips
  • When running mcmf in a Linux environment and the system prompts that there are not enough threads, use sysctl -w vm.max_map_count=11262144

How to use

Real-world platform

Coming soon

Simulation platform (Similar to real-world platforms)

Coming soon

Simulation Comparison Platform

1. Add environment variables to the system
   • export WORK_DIR=/your_path/pb_nbv/
2. Download test data and pt_scvp file
   • test_models. Unzip the models.zip file and place "hb_models", "lm_models" and "stanford_models" in the ${WORK_DIR}src/gazebo_benchmark_env/env_startup/models.
   • scvp_pt. Place the "last.pth.tar" in${WORK_DIR}src/scvp_core/sc-net/pt.
3. Generate the sdf file for Gazebo
   • Please make sure that "hb_models", "lm_models" and "stanford_models" are in the correct directory.
   • python3 src/gazebo_benchmark_env/env_startup/script/create_model_sdf.py
4. Compile the ROS workspace
   • cd ${WORK_DIR} && catkin_make
5. Run script
   • (mcmf) python3 src/gazebo_benchmark_env/env_startup/script/start_benchmark_mcmf.py
   • (nbvnet) python3 src/gazebo_benchmark_env/env_startup/script/start_benchmark_nbv_net.py
   • (pb) python3 src/gazebo_benchmark_env/env_startup/script/start_benchmark_pb.py
   • (scvp) python3 src/gazebo_benchmark_env/env_startup/script/start_benchmark_scvp.py
   • (see) python3 src/gazebo_benchmark_env/env_startup/script/start_benchmark_see.py

📚 BibTeX

If you use this repository for your research, please cite our RA-L paper:

@article{jia2025pb,
  title={PB-NBV: Efficient Projection-Based Next-Best-View Planning Framework for Reconstruction of Unknown Objects},
  author={Jia, Zhizhou and Li, Yuetao and Hao, Qun and Zhang, Shaohui},
  journal={IEEE Robotics and Automation Letters},
  year={2025},
  publisher={IEEE}
}