Speak-zee

How to Use?

Inspiration

As an all-student team, a very common struggle that all of us have faced at some point or the other is our lack of confidence, especially during high nerves situations such as presentations or live pitching. According to Psychology Today, "some 7 percent of the adult American population suffers from an extreme fear of public speaking is a subtype of social anxiety disorder". To a point, we figured out through research that stage fright doesn't simply affect students but is a grave social problem at large and that was the motivation behind building Speak-zee.

What it does

We have a browser-side recorder that can record you while you are practicing your speaking skills and help you out with real-time live feedback based on your speaking posture, voice confidence and whether you are speaking confidently on your own or using any external aids like paper, mobile phone, etc. It can also record your speech.

How we built it

We used EJS (template engine), CSS and client side Javascript on the front-end side
We used Bootstrap framework
We used Google Teachable Machine to train the pose, audio and image models and to produce live feedback for the speech.
We used Node.js on the back-end server side
We used socket.io to transmit audio from client to server
We used Deepgram API for speech to text transcription and WPM metric
We used Canva for graphics and logo.
We used Plotly for making our speech data based graph.
We used jsPDF for converting all the data and rendering it in the form of a .pdf speech report.

Challenges we ran into

It was really tough to use image, pose and audio models- all three at once in our client side javascript code.
We initially started with an altogether different challenge but had to change our ideas twice during such a short frame, so it was really tough to get fixed on our final idea.
We had a lot of issues trying to figure out how Deepgram works and to use it in our project.
We were having issues on how to use socket for sending audio from client to server especially since we hadn't done anything similar earlier.

Accomplishments that we're proud of

We are really happy that we could get something up in such a brief time, especially due to the fact that we had to change our minds on the product so many times.
One of our teammates also had to leave midway due to personal issues, so it was tough to shoulder the responsibilities at once.

What we learned

We learned to use Google Teachable Machine to help detect, train and manipulate real time recordings for image, pose and audio models.
We leaned to use Deepgram
We also learned a lot of remote collaboration effectively since we had team members attending from various time zones.
We also learned how to host a dynamic app using Heroku.

What's next for Speak-zee

We would love to refine our UI further
We would also look forward to adding more datasets thereby improving the precision of our speech analyzer app
We would also like to build a community around Speak-zee so that more students and job seekers or any aspiring speakers alike can use our app, collect feedback and benefit.