CoSpar: Online Learning from Human Preference and Coactive Feedback

This repository contains code for reproducing the CoSpar algorithm simulation results and plots in the paper:

Preference-Based Learning for Exoskeleton Gait Optimization
Maegan Tucker*, Ellen Novoseller*, Claudia Kann, Yanan Sui, Yisong Yue, Joel W. Burdick, and Aaron D. Ames
IEEE Conference on Robotics and Automation (ICRA), 2020
(*equal contribution)
PDF     Video

The CoSpar algorithm was experimentally deployed on a lower-body robotic exoskeleton to identify personalized, user-preferred exoskeleton walking gaits.  CoSpar is a human-in-the-loop learning framework that adapts to mixed-initiative qualitative feedback from a human (i.e., pairwise preferences and coactive feedback) to optimize an unknown objective function, for instance, an exoskeleton user's comfort.

Implementation Details

The code is divided into two subfolders, corresponding to the compass-gait biped simulations and the 2D synthetic function simulations, respectively. The Python files included are as follows:

1. Scripts titled Optimize_*.py contain the code for running the simulations.
2. Scripts titled Plot_*.py generate the plots in Section 4 of the paper.
3. Scripts titled Animate_*.py generate the image stacks used to make the animations in the video accompanying the paper.
4. Preference_GP_learning.py and CoSpar_feedback_functions.py contain helper functions called by 1), 2), and 3).
5. Generate_and_plot_2D_objective_functions.py contains the code for generating the 2D synthetic objective functions used in the 2nd set of simulations.
6. Illustrate_self_sparring.py makes some plots of posterior samples (for the compass-gait biped) that were used in the video accompanying the paper.

Descriptions of other folders:

1. The folders "Compass_gait_biped_simulations/Compass_biped_results/" and "2D function simulations/Sim_results/" contain the simulation results depicted in Figures 2-4 in the paper.
2. The folders "Compass_gait_biped_simulations/Plotting_data/" and "2D function simulations/2D_obj_1_sim_posteriors" contain results of computations performed by some of the plotting code (which processes data saved by the simulations), so that these quantities don't need to be re-computed each time that one runs the plotting scripts.
3. The folder "2D function simulations/Sampled_functions_2D/" contains the 2D synthetic objective functions used in the simulations; these are generated by the script Generate_and_plot_2D_objective_functions.py.