Progression Modelling for Online and Early Gesture Detection
Vikram Gupta, Sai Kumar Dwivedi, Rishabh Dabral and Arjun Jain
Oral Paper, 3D Vision, 2019
This repository contains the tightly annotated Neo-Nvidia annotations contributed in our paper. In this paper, we propose a multi-task framework for online and early gesture recognition which is crucial for building touchless gesture bases interfaces. In the online settings, our approach recognises the gestures at an early stage with high precision and also achieves state-of-the-art results on the NVIDIA gesture dataset in the offline settings. We also evaluated our approach on the Montalbano dataset and report competitive results.
The annotations have been shared in json format. Each entry represents the annotation for a video from the NVIDIA dataset with the category "gesture_id" and the start/end frames "gesture_spans" for the gesture. Following is an example of the annotation.
{
"gesture_spans":["85:106"],
"gesture_id":1,
"video_path":"class_01/subject2_r0"
}
Some videos have multiple gesture spans as multiple instances of the same gestures were performed in the video.
To cite our datasets, please use the following bibtex records:
@inproceedings{gupta2019progression,
title={Progression Modelling for Online and Early Gesture Detection},
author={Gupta, Vikram and Dwivedi, Sai Kumar and Dabral, Rishabh and Jain, Arjun},
booktitle={2019 International Conference on 3D Vision (3DV)},
pages={289--297},
year={2019},
organization={IEEE}
}
We thank Mallikarjun BR, Neha Tarigopula, Sanath Narayan and Shuaib Ahmed for their valuable feedback and their help in the annotation of the dataset.
We gratefully acknowledge Brijesh Pillai and Partha Bhattacharya at Mercedes-Benz Research and Development India, Bangalore for providing the funding and infrastructure for this work.
For any questions and contributions, you can contact the authors of the paper, whose emails are listed in the paper.
Disclaimer: This is not an official Daimler product.