Spell The Sign (ASL Edition)

An exploration of VR multi-interaction for fun, intuitive learning experiences.

Comment

Overview

Can you Spell the Signs? Spell the Sign delivers American Sign Language (ASL) education through fun, intuitive multimodal interactions. Our virtual reality (VR) venture comprises two exciting modes: Learning and Battle. In the Learning mode, participants engage with ASL through interactive sessions, mirroring the poses and gestures demonstrated by virtual hand models in a dynamic environment. Enter the Battle mode, where the excitement peaks. Participants blend voice interaction techniques with hand gestures to cast spells, engaging in a virtual combat experience against “fearful” enemies. Track: Ultraleap Hand Tracking for Entertainment & XR Trailblazer

Project Objectives

Beyond delivering a fun learning experience for all, we also aimed to explore intuitive, multi-modal player interactions. For this project, we used UltraLeap’s hand-tracking Unity SDK (for both static hand poses and the Oculus Voice SDK to enhance the interactive experience of the users. We want to leave behind the point-and-click system that VR applications often use, and consider more ways to access the world of VR.

How we built it & build details

Hardware: UltraLeap Leap Motion controller, Quest 2 / Quest 3 Target Platform: XR2 devices

Assets Used: Low-poly island (Unity Free Asset), Skybox (Unity Free Asset), ASL Alphabet Icons (Images were taken from internet, cards/sprites were re-designed in Figma & Unity), Enemy Skeleton, (Unity Free Asset), Enemy Spawn Portal (Unity Free Asset)

Challenges

  • Early on we had some difficulties with getting the SDKs to work
  • Due to the larger scope of our project, we ended up with a really large file that hindered the speed for building & debugging

What's next for Spell The Sign (ASL Edition)

  1. More Lessons
  2. More interactive modes?
  3. Changing scenes / ramping up the art and design even more.

Built With

Share this project:

Updates