This is a picture of our whiteboard during the Sandbox hackathon. It's one of my favorite pictures because a story of devotion, passion, and creativity lies behind it.
Before the hackathon, I noticed a big pain among deaf/HH people is the inability to feel included in vocal conversations. Current TTS/STT services offer very basic assistance, but in the current day and age, deaf people deserve much more.
After 8 hours of intense coding fueled by energy drinks, pizza, and our desire to help, we finally finished building SpeakUp.SpeakUp is an app that converts real-life conversations into a group chat in real-time. It uses a deep learning technology called Speaker Diarization that segments speech into its respectable speaker before converting it into text in real-time. This way, deaf people don't feel isolated in fast and unpredictable conversations with hearing people.
It also converts speech to text with an auditory "typing" status to prevent interruptions.
Building such an app from scratch has been an inspiring learning experience and it couldn't be done without the brilliant engineers Cody Arvonen and Sonny George! Seriously talented and a pleasure to work with.
Increase risk of depression for Deaf/HoH people
Speaker segmentation Identification accuracy
Hours took to develop such a model from scratch (Developed in Hackathon)
survey responses gathered in 4 days from target audiences for customer research and product validation
Speaker segmentation accuracy
