MCVO

Robust Visual Odometry using Rigidly-Bundled Arbitrarily-Arranged Multi-Cameras

Making multi-camera visual SLAM systems easier to set up and more robust to the environment is attractive for vision robots. Existing monocular and binocular vision SLAM systems have narrow sensing Field-of-View (FoV), resulting in degenerated accuracy and limited robustness in textureless environments. Thus multi-camera SLAM systems are gaining attention because they can provide redundancy with wider FoVs. However, the usual arbitrary placement and orientation of multiple cameras make the pose scale estimation and system updating challenging. To address these problems, we propose a robust visual odometry system for rigidly-bundled arbitrarily-arranged multi-cameras, namely MCVO, which can achieve metric-scale state estimation with high flexibility in the cameras' arrangement. Specifically, we first design a learning-based feature tracking framework to shift the pressure of CPU processing of multiple video streams to the GPU. Then we initialize the odometry system with the metric-scale poses under the rigid constraints between moving cameras. Finally, we fuse the features of the multi-cameras in the back-end to achieve robust pose estimation and online scale optimization. Additionally, multi-camera features help improve the loop detection for pose graph optimization. Experiments on KITTI-360 and MultiCamData datasets validate its robustness over arbitrarily arranged cameras. Compared with other stereo and multi-camera visual SLAM systems, MCVO obtains higher pose estimation accuracy with better robustness. This code runs on Linux, and is fully integrated with ROS.

Videos:

1. Prerequisites

1.1 Ubuntu and ROS Ubuntu 20.04. ROS Noetic. ROS Installation additional ROS pacakge

    sudo apt-get install ros-YOUR_DISTRO-cv-bridge ros-YOUR_DISTRO-tf ros-YOUR_DISTRO-message-filters ros-YOUR_DISTRO-image-transport

1.2 Ceres Solver Follow Ceres Installation, use version 2.0.0 and remember to sudo make install.

1.3 PCL Follow PCL_Installation, use version 1.10.0

1.4 OpenCV Follow OpenCV_Installation, use version 4.5.0

    sudo apt update && sudo apt install -y cmake 
    
    mkdir ~/catkin_ws/ThirdParty && cd ~/catkin_ws/ThirdParty
    
    git clone https://github.com/opencv/opencv.git
    git clone https://github.com/opencv/opencv_contrib.git
    cd opencv
    git checkout 4.5.0
    cd ../opencv_contrib
    git checkout 4.5.0
    cd ../opencv
    mkdir build && cd build
    
    cmake -D CMAKE_BUILD_TYPE=RELEASE   -D CMAKE_INSTALL_PREFIX=../../opencv/install -D CMAKE_BUILD_TYPE=RELEASE  -D WITH_OPENGL=ON       -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules ..
    
    make install -j 4

Or

    sudo apt-get install libopencv-dev=4.5.0*

1.5 Eigen Follow OpenCV_Installation, use version 3.3.7

    apt-get install wget unzip
    cd ~/catkin_ws/ThirdParty  
    wget https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.tar.gz
    tar zxf eigen-3.3.7.tar.gz
    mv eigen-3.3.7.tar.gz eigen

Or

    sudo apt-get install libeigen3-dev=3.3.7-1

2. Build MCVO on ROS

Clone the repository :

    cd ~/catkin_ws/src
    git clone https://github.com/JunhaoWang615/MCVO.git

Put model and initial_image in src/MCVO/MCVO/config

Put brief_k10L6.bin in src/MCVO/support_files

catkin_make

    cd ../
    catkin_make
    source ~/catkin_ws/devel/setup.bash

3. Visual Odometry and Loop Closure on Public datasets

Download KITTI360 Dataset. and then convert it to .bag file. The dataset has a total of 4 cameras, including two forward-looking cameras and one fisheye camera on the left and right. The system also works with MultiCamData dataset(MultiCamSLAM). We take KITTI360 sequence00 0-2277 frames as the example.

3.1 (Optional) Select the feature extraction algorithm and the cameras you wish to run in src/MCVO/MCVO/config/KITTI360/KITTI360.yaml. You can also modify KITTI360.yaml to another yaml file in src/MCVO/MCVO/launch/KITTI360.launch to fit your dataset

3.2 Open three terminals, launch the mcvo_estimator , rviz and play the bag file respectively. Take KITTI360 sequence00 0-2277 frames for example

    roslaunch mcvo KITTI360.launch 
    rviz -d src/MCVO/MCVO/launch/KITTI360.rviz
    rosbag play YOUR_PATH_TO_DATASET/KITTI360_00_1.bag