CoSpar: Preference-Based Learning for Exoskeleton Gait Optimization

Overview

CoSpar is a preference-based learning method. It is based on the SelfSparring algorithm and allows users to actively provide coactive feedback while accepting preference feedback.

This repository provides an implementation of the CoSpar algorithm proposed in the following paper:

Preference-Based Learning for Exoskeleton Gait Optimization
Maegan Tucker, Ellen Novoseller, Claudia Kann, Yanan Sui, Yisong Yue, Joel Burdick, Aaron D. Ames
International Conference on Robotics and Automation (ICRA), 2020
PDF

Demo

Installation

Python = 3.13

pip install -r requirements.txt && pip install -e .

Usage

To get started, it is recommended to run the interactive notebooks in the examples folder. We provide 1-d and 2-d examples for the algorithm.

An example for 1-d:

import torch
from cospar import CoSpar

# Initialization
candidates = torch.linspace(0, 10, 100).unsqueeze(-1)
bounds = torch.tensor([[0.], [10.]])
datapoints = candidates[[40, 60]]
value, comparisons = func_with_compare(datapoints)
fmin = torch.tensor([0.2])
lipschitz = torch.tensor([0.5])

algo = CoSpar(
    candidates=candidates,
    bounds=bounds,
    datapoints=datapoints,
    comparisons=comparisons,
    num_samples=2,
    buffer_size=0,
    covar_module=None,
    ci_beta=2.0,
    enable_fit=False,
)

# Optimization
for _ in range(20):
    next_point = algo.next()
    if next_point is None: break
    _, pairwise_comparison = func_with_compare(next_point)
    preference_point = get_coactive_feedback()
    new_datapoints = torch.cat([next_points, preference_point], dim=0)
    coactive_comparison = get_coactive_comparison()
    algo.update(new_points, pairwise_comparison, coactive_comparison)

1. For how to construct comparisons, please refer to Pairwise GP Models.
2. You can specify the coactive_weight parameter of the CoSpar algorithm, which defaults to 1.0.

BibTeX

@INPROCEEDINGS{9196661,
  author={Tucker, Maegan and Novoseller, Ellen and Kann, Claudia and Sui, Yanan and Yue, Yisong and Burdick, Joel W. and Ames, Aaron D.},
  booktitle={2020 IEEE International Conference on Robotics and Automation (ICRA)}, 
  title={Preference-Based Learning for Exoskeleton Gait Optimization}, 
  year={2020}
}