Inclusive Design, Product Design

Converse is an application that helps deaf people to communicate with others at work or personal life in a way that creates deep and meaningful connections.

Client: Student Project
Year: 2019
My Role: Inclusive Designer, User Researcher, Prototyper
Tools: Sketch, Invision, Photoshop, Pen & Paper


Methods: Participatory Design, Storyboarding, Sketching, Prototyping, User Testing

Lori Leal was born deaf and learned sign language at 14 years old. She received family support to strengthen her comprehension and vocabulary however, she still struggled with daily communication. The barrier to communicate with hearing people at work and in her personal life prevents her from forming deep and meaningful relationships in which she feels isolated and alone


Our participatory design activity was divided into five phases where we introduced design dimensions, technologies, and ideas available for use. In each phase, Lori picked aspects of the design she wanted to pursue. This paced the introduction of new factors so she could consider them individually before integrating them into a larger whole. It also built ownership in the design solution since she had picked all of them. We think this part of our process was very robust and had the intended effect.

maxresdefault copy


The team conducted an interview and brainstorming session with Lori to define a design problem or opportunity. Our interview session consisted of covering the basics of participatory design; the goal, the scope of the project, personal introductions and the structure of the interview. Then we asked Lori questions about her daily routine, work, family, her hobbies and what she did to enjoy herself. We focused on identifying specific points where there were unmet needs or problems without a solution that could be solved to improve the quality of her life and asked follow up questions to gain more insight into why.

Lori mentioned two interrelated problems: a lack of deep meaningful communication, which resulted in feelings of isolation and loneliness. We briefly reviewed the technology she used to currently overcome this problem and then talked more about the specific challenges to communication that prevented her from forming communication.



Our project focuses on alleviating the participant’s feelings by promoting deep meaningful communication. The specific findings from our interview with Lori that led to this focus are as follows:

1- Challenges in conversing and connecting with others
●      Lori’s family does not sign and is the only deaf person in her family.
●      She is trained to lip-read, however, she usually grasps an eighth of all words spoken.
●      Lori avoids communication with hearing people as it is a frustrating experience. 
●      She is willing to ask people to write but it is often problematic or not possible. 

2- Feeling isolated
●      Lori worked at her previous job for 18 years but had no direct communication with anyone due to the communication barrier and not being able to hear them.
●      She feels that using an interpreter for communication does not create a connection between them and another person since the interpreter is viewed as a proxy.
●      While traveling solo, Lori noticed that people try to talk to her but then shy away when they realize that she is deaf.
●      Even if having a conversation, it remains surface-level and not deep.

sketches 4


Based on the problems we identified in our first interview, it is clear that Lori’s biggest daily challenge is communication. The barrier to communication with hearing people at work and in her personal life prevents from forming deep meaningful relationships without which they feel isolated and alone. Therefore, in this project, we will focus on creating solutions that help her communicate with others in a way that creates the deep and meaningful connections that Lori seeks.



During the data analysis process, four key tasks were defined that our participant may want to complete with the technology we were designing together. The tasks are independent of any specific technology so that we could still explore many designs. Through sketching, we brainstormed a range of design ideas for how to support those tasks and considered a mix of different platforms, like physical interactive objects, smartphones, tablets, and laptops may be helpful. 




1. Converse independently: a) Talk to someone without an interpreter b) the Deaf person wants to respond to a conversation (family gathering, conversation at the workplace), or make quick comments on a fast-paced group conversation.

2. Invite others to or act on opportunities to converse: a) Invite or act on opportunities to converse with new people b) Responding to situational conversation (on the street, or shopping, or while hiking) while ordering for a simple need and improve the ability of communication (ordering food, ordering drinks …)

3. Having a deep one-on-one or group conversation: a) Have a nuanced (adopting a position on a topic, expressing an opinion, etc.) conversation discussion with a co-worker or family member. b) Deaf person wants to respond to a conversation (family gathering or meeting), or make quick comments on a fast-paced group conversation

4. General conversation skills: like social confidence, comprehension, etc.



  • Avatar or hologram which translates sign language to speech, and vice versa
  • ASL to speech translation system
  • Real-time text communication where both people can type at the same time.
  • Smart contacts with live captioning feature
  • 360-degree camera with live captioning and text to speech system
  • ...
sketches 9
sketches 10
sketches 5
Screen Shot 2019-12-23 at 9.38.24 PM
Screen Shot 2019-12-23 at 9.37.51 PM
Screen Shot 2019-12-23 at 9.38.09 PM




We tested out the protocol we made with some pilot participants to make sure specific points, questions, and materials are working and modify the errors.



  • Review Goals, Progress, and Team’s Design Ideas:
    • Review of the project and design goal that was set during the first interview
    • Explain the co-design session: create a design using Lori's lived experience
    • Present design ideas in the form of sketches, storyboards, and prototypes
    • Ask her to think out loud while going through the design materials
  • Co-design Activity w/ Participant:
    • Distribute design materials (paper, markers, etc.) to Lori and the team
    • Emphasize not to bound by any of the previous ideas, and can sample the best parts to create a new idea, or work off an existing idea if they wish
    • Introduce co-design activity
    • Start with storyboarding. Work with the Lori to come up with a storyboard that represents a nuanced need that she has
    • Based on the storyboard, everyone creates initial sketches and prototypes.
    • Encourage divergent thinking. Make note of all idea threads Lori generates


  • Discussion and New Design Direction:
    • Discuss the new design artifacts from the co-design activity and critique it
    • Create a vision where this design is completed and might be used in the wild
    • Wrap Up and Next Steps: Thank Lori. Inform her of completing the final design and prototype testing
Screen Shot 2019-12-23 at 9.36.51 PM
sketches 7


Considering Lori’s insights from the co-design session, we built a device including a 360° camera and an accompanying app called Converse, which would provide meaningful communication within desired social environments. The camera captures the live video of all people in the conversation and the application shows a waist-up view of all people on the top half of the mobile screen, with a larger image of the person talking. 


Paper Prototype


Interaction Design & Visual Design Iterations

The live translation of the conversation overlays on the bottom half of the screen with the name of people. Lori responds to the conversation either through Deaf speech or by typing on a text box in the far bottom, which will be turned into speech for the audience. In the future, when deaf speech to text technology is commercially viable, Lori will be able to speak back to the device instead of having to type. This created our project focus to encourage inclusiveness and reduce isolation within a social environment.



We began the session by demonstrating some of the major changes and critiques from our co-design session to spark comments and insights. Then, we observed Lori while interacting with the prototype and asked questions to understand her satisfaction from the design concepts within the app and how it would function if it was used in a real-life situation.

Final participant comments and insights

One of the major improvements was the app’s ability to detect and display background noise. Lori stressed that without the ability to hear, there is a lack of access to the noise that is occurring around them. The background noise detection within the app would provide an understanding and an awareness that is lacking for a Deaf person within a hearing environment. She also appreciated the ability to change the sensitivity of the notification to alert any emergencies occurring around her.



We also provided a method of how the background noise could be notified, by a banner, within the conversation chat, or pop-up notification. Lori enjoyed seeing the notification embedded right into the chat and the only critique was to change the color so it would stand out from the conversation. She also appreciated that when the background notification was clicked, it would open a new window that would show intervals of past noise from 15 minutes ago, 10 minutes ago and what was picked up moments ago.


Lori loved the ability to customize the chat features such as changing the font size, change the chat speed, changing the color of the text, and changing the color of the background. For instance, someone who was colorblind was using the app, they would be able to access the chat by customizing it to their liking and ability. She really enjoyed that there was a microphone conveniently located on the main screen to utilize Deaf speech to text instead of typing. Lori also wanted the ability to control the conversation in her favor.
She wanted to be able to focus on certain people within the social environment. This would occur in moments of Lori using the app and desires to switch the large image to focus on someone else who she prefers to have a conversation with; this would give her some control over the device and who it focuses on. She also appreciated that we acknowledged that typing was a last resort.