DexDrummer: In-Hand, Contact-Rich, and Long-Horizon Dexterous Robot Drumming

Official implementation of DexDrummer by Hung-Chieh Fang, Amber Xie, Jennifer Grannen, Kenneth Llontop, and Dorsa Sadigh.

[Paper] [Website]

Installation

conda env create -f environment.yml
conda activate dexdrummer

Data

The repository already includes example MIDI sheets in data/, including:

• everlong_25_80.mid
• seven_nation_army_35_80.mid

Download soundfont file

wget https://keymusician01.s3.amazonaws.com/FluidR3_GM.zip

Quick Start

1) Uni-manual training (simulation)

Reference script: scripts/train_uni_cclm.sh

bash scripts/train_uni_cclm.sh <exp_name> <drum_sheet_name> <max_episode_steps> <hand>

Example:

bash scripts/train_uni_cclm.sh test seven_nation_army_35_80 200 left

2) Uni-manual evaluation

Reference script: scripts/eval_uni.sh

bash scripts/eval_uni.sh <exp_name> <drum_sheet_name> <max_episode_steps> <hand>

3) Bi-manual evaluation

Reference script: scripts/eval_bi.sh

bash scripts/eval_bi.sh <left_exp_name> <right_exp_name> <drum_sheet_name> <max_episode_steps>

Flags used in the scripts:

• --arm_mp: enables arm motion planning from retargeted stick trajectories.
• --arm_res: enables residual RL for the arm.
• --drum_set hihat_snare: two-drum setup. Use whole_set for full kit training/evaluation.

Sim2Real Deployment

Real-world deployment and ROS integration code live in real/src.

1) Choose a trained checkpoint

Set the run name in real/src/dg5f_driver/script/rl_policy_node.py:

• PolicyConfig.run_name: folder under runs/ to load.
• PolicyConfig.root: repository root path on your machine.
• PolicyConfig.enable_left / PolicyConfig.enable_right: enable the hand(s) you want.
• PolicyConfig.do_open_loop: open-loop replay (True) vs closed-loop policy (False).

2) Build ROS workspace

colcon build
source install/setup.bash

3) Start robot-side services (separate terminals)

Launch only the hands you are using:

ros2 launch dg5f_driver pid_left_driver.launch.py
ros2 launch dg5f_driver pid_right_driver.launch.py
python3 -m real.src.dg5f_driver.script.dg5f_interpolate

4) Run policy node

python3 -m real.src.dg5f_driver.script.rl_policy_node

5) Sim2Sim dry-run (before hardware)

In real/src/dg5f_driver/script/rl_policy_node.py, set _ROS2_AVAILABLE = False to run the same policy loop without ROS/hardware. This is useful for checking trajectory quality and comparing sim vs real behavior.

Citation

@misc{fang2026dexdrummer,
      title={DexDrummer: In-Hand, Contact-Rich, and Long-Horizon Dexterous Robot Drumming}, 
      author={Hung-Chieh Fang and Amber Xie and Jennifer Grannen and Kenneth Llontop and Dorsa Sadigh},
      year={2026},
      eprint={2603.22263},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2603.22263}, 
}