RadarHand
RadarHand is a wrist-worn wearable system that uses radar sensing to detect on-skin proprioceptive hand gestures, making it easy to interact with simple finger motions. Radar has the advantage of being robust, private, small, penetrating materials and requiring low computation costs.
In this project, we first evaluated the proprioceptive nature of the back of the hand and found that the thumb is the most proprioceptive of all the finger joints, followed by the index finger, middle finger, ring finger and pinky finger. This helped determine the types of gestures most suitable for the system.
Next, we trained deep-learning models for gesture classification. Out of 27 gesture group possibilities, we achieved 92% accuracy for a generic set of seven gestures and 93% accuracy for the proprioceptive set of eight gestures. We also evaluated RadarHand’s performance in real-time and achieved an accuracy of between 74% and 91% depending if the system or user initiates the gesture first.
This research could contribute to a new generation of radar-based interfaces that allow people to interact with computers in a more natural way.
Project Video(s):https://www.youtube.com/watch?v=BxbbxNes9vI
Publications
-
RadarHand: a Wrist-Worn Radar for On-Skin Touch-based Proprioceptive Gestures
Hajika, R., Gunasekaran, T. S., Haigh, C. D. S. Y., Pai, Y. S., Hayashi, E., Lien, J., ... & Billinghurst, M.Hajika, R., Gunasekaran, T. S., Haigh, C. D. S. Y., Pai, Y. S., Hayashi, E., Lien, J., ... & Billinghurst, M. (2024). Radarhand: A wrist-worn radar for on-skin touch-based proprioceptive gestures. ACM Transactions on Computer-Human Interaction, 31(2), 1-36.
@article{hajika2024radarhand,
title={Radarhand: A wrist-worn radar for on-skin touch-based proprioceptive gestures},
author={Hajika, Ryo and Gunasekaran, Tamil Selvan and Haigh, Chloe Dolma Si Ying and Pai, Yun Suen and Hayashi, Eiji and Lien, Jaime and Lottridge, Danielle and Billinghurst, Mark},
journal={ACM Transactions on Computer-Human Interaction},
volume={31},
number={2},
pages={1--36},
year={2024},
publisher={ACM New York, NY, USA}
}We introduce RadarHand, a wrist-worn wearable with millimetre wave radar that detects on-skin touch-based proprioceptive hand gestures. Radars are robust, private, small, penetrate materials, and require low computation costs. We first evaluated the proprioceptive and tactile perception nature of the back of the hand and found that tapping on the thumb is the least proprioceptive error of all the finger joints, followed by the index finger, middle finger, ring finger, and pinky finger in the eyes-free and high cognitive load situation. Next, we trained deep-learning models for gesture classification. We introduce two types of gestures based on the locations of the back of the hand: generic gestures and discrete gestures. Discrete gestures are gestures that start at specific locations and end at specific locations at the back of the hand, in contrast to generic gestures, which can start anywhere and end anywhere on the back of the hand. Out of 27 gesture group possibilities, we achieved 92% accuracy for a set of seven gestures and 93% accuracy for the set of eight discrete gestures. Finally, we evaluated RadarHand’s performance in real-time under two interaction modes: Active interaction and Reactive interaction. Active interaction is where the user initiates input to achieve the desired output, and reactive interaction is where the device initiates interaction and requires the user to react. We obtained an accuracy of 87% and 74% for active generic and discrete gestures, respectively, as well as 91% and 81.7% for reactive generic and discrete gestures, respectively. We discuss the implications of RadarHand for gesture recognition and directions for future works.