Skip to content

Yash1361/SoundScape

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

To build our project, we developed an iOS-based app using Swift, designed to assist visually impaired individuals in locating objects and navigating complex indoor environments. By leveraging advanced technologies like augmented reality, machine learning, spatial audio, and voice recognition, we created a highly sophisticated and technically challenging solution. The app uses Swift's ARKit and the iPhone's LiDAR Scanner to detect objects and map the surroundings accurately. This setup allows for precise detection of surfaces, including vertical, horizontal, and angled planes, enabling the app to guide users effectively through spaces. For object detection, we integrated the YOLOv3 model with CoreML, allowing the app to identify various items, such as people, bottles, bags, and phones, in real-time. The integration of YOLOv3 with Swift posed significant technical challenges, as we needed to ensure that the object detection was both fast and accurate, even in dynamic environments. A standout feature of the app is its use of spatial audio via AirPods Pro. This allows the app to emit directional sounds, guiding the user towards objects and helping them navigate by adjusting the volume based on proximity—the closer the obstacle or desired object, the louder the sound. This required precise calibration of audio cues with real-time object positions, ensuring accurate and intuitive feedback. One of the app's most complex aspects is its dynamic 3D mapping and object tracking. As the user moves, the app continuously calculates and updates the positions and distances of objects relative to the user, even if objects move out of view. This continuous mapping and tracking create a reliable reference for navigation, demanding efficient data handling and processing to maintain performance and accuracy. When a user searches for a specific object, the app places a virtual beacon on the detected item and provides proximity feedback through spatial audio cues, which become louder as the user approaches the object. The development of this app required the seamless integration of multiple advanced technologies, pushing the boundaries of iOS development. The challenges included real-time performance optimization, precise spatial audio calibration, and the dynamic adaptation of 3D mapping in constantly changing environments. By overcoming these hurdles, we created a powerful tool that significantly enhances the independence and mobility of visually impaired users.

About

An iOS app enabling safe indoor navigation for the visually impaired using real-time 3D mapping, object detection, and spatial audio guidance.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages