$\textsf{\color{orange}{M}\color{orange}{P}\textcolor{orange}{1}}$: $\textsf{\color{orange}{M}}\text{eanFlow}$ Tames $\textsf{\color{orange}{P}}\text{olicy}$ Learning in $\textsf{\color{orange}{1}}$-step for Robotic Manipulation

Project Page

Training & Deployment of R-MP1

More info:

• Robot: ARX R5;
• Camera: Realsense L515;

💻 Installation

1. # install ros1 
- wget http://fishros.com/install -O fishros && . fishros   // install ros
2. # install librealsense-2.54.2
- sudo apt-get install git libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev
- git clone https://github.com/IntelRealSense/librealsense/releases/tag/v2.54.2
- cd librealsense-2.54.2
- ./scripts/setup_udev_rules.sh
- mkdir build
- cd build
- cmake ..   
- make -j4
- sudo make install
3. # test
- realsense_viewer

🤖 Launch Robotics

1. Setting robotic can rules.

0. git clone https://github.com/LogSSim/R-MP1.git
1. cd R-MP1, rm -rf build devel, catkin init, catkin build, roscore 
2. source devel/setup.bash
// setup，Power the robotic arm’s port as instructed；
3. rosrun arx_pkg setup_can_node.py    
4. rosrun arx_pkg start_can_node.py

2. Launch demonstrator.

// new terminal;
1. source devel/setup.bash
2. ./calibration_robot.sh gripper // Calibrate the robotic arm; usually only the gripper needs calibration—set it to position 0 first, and do not run this script alongside the subsequent launch command.
3. roslaunch arx_pkg master_and_follower.launch

📚 Data collection

1. Installation: cd src/Brain/MP1

0. Install MP1 environment. install.md

1. useconda
2. conda activate rmp1
3. python tools/multi_realsense.py // check your point cloud.
4. python collect_points_cloud.py --save_dir datasets/task/  // press x-box Y to start collecting, A to finish a episode, X to exit collection.

2. Data: Check your data

1. chmod +x points_cloud_tools/*.py
2. python3 points_cloud_tools/visualize_zarr_pointcloud.py datasets/heat-water/episode0.zarr

3. Merge data.

python ./points_cloud_tools/merge_zarr_data.py datasets/heat-water/ datasets/train/

🛠️ Usage

Robot. Ctrl+c to close the roslaunch arx_pkg master_and_follower.launch and open

roslaunch arx_pkg inference.launch

Train. The script to train policy, modify dataset dir in (src/Brain/MP1/RMP1/MP1/mp1/config/task/close-drawer_3d.yaml) and (src/Brain/MP1/scripts/train_policy.sh):

bash scripts/train_policy.sh mp1 heat-water_3d

Deploy. After you have trained the policy, change the checkpoint path in config file (src/Brain/MP1/RMP1/MP1/mp1/config/mp1.yaml), then deploy the policy with the following command.

# 3d policy
bash scripts/deploy_policy.sh mp1 heat-water_3d