HEAD RL training codebase!

Official implementation of RL training part for Hand Eye Autonomous Delivery: Learning Humanoid Navigation, Locomotion and Reaching Paper and Website.

Install

1. Install Isaac Gym following link
2. Install this project and dependencies

cd HEAD_rl_training
pip install -e .

3. Make data and output folder:

cd tml_humanoid
mkdir -p data/AMASS
mkdir exp_out

Download data from HERE Please put both files under data/AMASS

Run training

cd tml_humanoid/src_train/sim_rl
python run_training.py

Evaluate policy in Isaac Gym

cd tml_humanoid/src_train/sim_rl
python run_training.py --test

Change Configs

1. Default configs (30Hz control frequency) are inside head_and_hand_local_29.py policy trained after 30000 epochs should be sim2real ready
2. For 50Hz control frequency, config is head_and_hand_local_29_50hz.py haven't tested sim2real

Contribution

1. Pei Xu built the foundation of this repo, please checkout original repo ICCGAN
2. Zi-Ang Cao and Sirui Chen make this repository sim2real ready
3. Yanjie Ze Provide useful suggestions on parameter setting and reward tuning

Citation

@article{chen2025hand,
  title={Hand-Eye Autonomous Delivery: Learning Humanoid Navigation, Locomotion and Reaching},
  author={Chen, Sirui and Ye, Yufei and Cao, Zi-Ang and Lew, Jennifer and Xu, Pei and Liu, C Karen},
  journal={arXiv preprint arXiv:2508.03068},
  year={2025}
}