Translating sign language into text or speech.
Year:
2025
Timeframe:
4 months
Tools:
Figma, Miro
Category:
Emerging Technology
Overview
Using wearable sensors and intelligent sign language recognition, CoLingo provides a user-friendly and reliable solution for daily communication.
Aim
This project aims to create an offline, AI-powered mobile application that integrates with a smart glove to enable effective communication between DHH individuals and hearing individuals by translating sign language into text and speech, detecting emotional and stress states, and facilitating real-time speech-to-text conversion via an intelligent, context-aware software-based collaborative robot.
Project Info
Role: UX Research & Product Design Duration: 4 Months Tools: Figma, Miro
Methodology: Lean UX
I conducted 6+ user interviews and storytelling sessions with the people who are deaf and mute to understand their experiences and life challenges in simple daily tasks.
What users wants to say?
Modern AI models can now understand complete phrases instead of just single signs.
It use signals like heart rate and body reactions to understand stress or emotions, helping the technology respond more empathetically.
Most sign language systems are designed for ASL, which means they often fail to support other languages like BSL or ISL and may not recognize different personal signing styles.
I created three user personas to better understand different perspectives and experiences. The first persona represents someone born deaf, sharing what daily life and communication challenges are like. The second persona is a caregiver or guardian of a Deaf or mute individual, describing their experiences supporting and communicating with them. The third persona represents someone who has temporarily lost hearing or speech due to illness or medical conditions, highlighting the challenges of adapting to sudden communication barriers.
Who are we designing for, and what roles shape their experience?
After conducting user interviews and creating personas, I moved on to defining both user needs and business goals. Based on the insights gathered, I listed key assumptions about the users and the product and prioritized them to focus on the most important problems first.
To structure the ideas, I used tools like a Lean UX Canvas to outline the problem, users, solutions, and expected outcomes. I also created testable hypotheses and prioritized them to decide what should be validated first. Finally, I used the MoSCoW prioritization method (Must have, Should have, Could have, Won’t have) to identify the most critical features for the initial product.
From Research to Product Strategy
Lo-fi to High-fi
In the future, this project can be expanded to support the Deaf and Hard of Hearing community even more. I plan to introduce community forums where DHH users and interpreters can connect, ask questions, and communicate with each other. The platform could also allow users to share their personal experiences, helping others learn and feel supported.
Additionally, a learning hub can be added where users can explore resources related to sign language and communication. With the help of AI-powered recommendations, the app could suggest personalized learning content to help users improve their communication skills over time.
