Inclusive Design, Product Design

In a participatory design project and considering our user's insight, the Converse device was designed to provide deaf users to have meaningful communication within desired social environments.

Client: Student Project
Year: 2019
My Role: Inclusive Designer, User Researcher, Prototyper
Tools: Sketch, Invision, Photoshop, Pen & Paper


The Challenge

  • Bulidng relationship with hearing people
  • Daily communications at work and personal life
  • Feeling of isolation and loneliness

The Solution

Converse is an application that helps deaf people to communicate with others at work or personal life in a way that creates deep and meaningful connections.


The design dimensions, technologies, and ideas were introduced during the 5 phases of participatory design activity. In each phase, Lori picked aspects of the design she wanted to pursue, so it:

  • Paced the introduction of new factors so she could consider them individually before integrating them into a larger whole.
  • Built ownership for Lori in the design solution.
maxresdefault copy


Lori Leal was born deaf
Learned Sign language at age 14
Strong comprehension and vocabulary

Phase 1
Learn About Lori

About Interview

Identifying unmet needs or problems without a solution.
Defining design ideas and opportunities. 

Interview Focus

Daily routine
Work life
Family life
Hobbies & activities

Interview Insights

Major findings

Lori has been experiencing challenges in conversing and connecting with others which have caused feelings of isolation and loneliness.

Personal Life

  • Lori’s family does not sign and is the only deaf person in her family.
  • She is trained to lip-read but grasps an eighth of all words spoken.
  • She views an interpreter as a proxy who does not help to create a connection between her and people

Work life

  • Working in a job for 18 years, she had no direct communication with anyone due to the communication barrier.
  • Even if having a conversation, it remained surface-level and not deep.

Social Life

  • While traveling solo, people try to talk to her but then shy away when they realize that she is deaf.
  • She is willing to ask people to write but it is often problematic or not possible.  

Problem Space

The major problems and need found from the research

  1. Communication barriers
  2. Lack of deep and meaningful relationships
  3. Feeling isolated and alone

Construct point of view based on Lori’s needs

During the data analysis process, four key tasks were defined that Lori may want to complete with the design. The tasks are independent of any specific technology so that we could still explore many designs.




1. Converse independently
Talk to someone without an interpreter.
Respond to a conversation at life or work or make quick comments on a fast-paced group conversation.

2. Invite people to converse
Act on opportunities to converse with new people.
Respond to a situation  (e.g. ordering food).



3. Have a deep one-on-one or group conversation
Have a nuanced comversation (adopting a position on a topic, expressing an opinion, etc.)
Conversation discussion with a co-worker or family member.

4. General conversation skills 
Like social confidence, c
omprehension, etc.



Through sketching, we brainstormed a range of design ideas for how to support the above key tasks and considered a mix of different platforms, like physical interactive objects, smartphones, tablets, and laptops may be helpful.

sketches 4

Avatar or hologram which translatessign
to speech, and vice versa

sketches 7

360-degree camera with live captioning and text to speech system

Screen Shot 2019-12-23 at 9.38.24 PM

 Smart contacts with live captioning feature

Screen Shot 2019-12-23 at 9.37.51 PM

ASL to speech translation system

Screen Shot 2019-12-23 at 9.38.09 PM

Real-time text communication where both people can type at the same time.


Real-time text communication where
both people can type at the same time.



We tested out the protocol we made with some pilot participants to make sure specific points, questions, and materials are working and modify the errors.


Co-Design Session With Lori



Co-Design Protocol

  1. Review design goals, progress, and team’s design ideas
  2. Present design ideas in the form of sketches, storyboards, and prototypes
  3. Create one design/solution using Lori's lived experience

  4. Have Lori to think out loud while going through the design materials

Co-Design Activity

We distributed design materials (paper, markers, etc.) to Lori and the team and emphasized not to bound by any of the previous ideas, just can sample the best parts to create a new idea, or work off an existing idea if they wish.

Started with storyboarding to represent a nuanced need that she has
Based on the storyboard, everyone created initial sketches and prototypes.
Encourage divergent thinking. Make note of all idea threads Lori generates


Discussion and New Design Direction:

  • Everyone shared designs and soluctions
  • Lori and team picked one design idea
  • The new design artifacts from the co-design activity were discussed and created
  • Critiqued a vision where this design is completed and might be used in the wild
Screen Shot 2019-12-23 at 9.36.51 PM


Design Decision

Considering Lori’s insights from the co-design session, we designed a device including a 360° camera and an accompanying app called Converse, which would provide meaningful communication within desired social environments.




Paper Prototype


Selected Features

Defining the Functionalities


  • The camera captures the live video of all people in the conversation and the application shows a waist-up view of all people on the top half of the mobile screen, with a larger image of the person talking. 
  • Videos of all people are clickable and focusable, so Lori would switch to someone else who she prefers to have a conversation with.
  • The live translation of the conversation overlays on the bottom half of the screen with the name of people.
  • Lori responds to the conversation either through Deaf speech (using the microphone on the main page) or by typing on a text box in the far bottom, which will be turned into speech for the audience.
  • Ability to customize the chat features such as changing the font size, change the chat speed, changing the color of the text (for colorblind users), and changing the color of the background.

Interaction Design & Visual Design Iterations



Evaluation & Walkthroughs

The formal testing was done with the InVision prototype and we observed Lori while interacting with the prototype to complete the test tasks. We also asked questions to understand her satisfaction from the design concepts within the app and how it would function if it was used in a real-life situation.


Key Findings from the Testing

Overall, Lori was very positive toward the prototype. She did not have issues in completing tasks and appreciated the simplicity of design.
One of the major improvements was the app’s ability to detect and display background noise. Lori stressed that without the ability to hear, there is a lack of access to the noise that is occurring around them. The background noise detection embedded right into the chat would provide an understanding and an awareness that is lacking for a Deaf person within a hearing environment. The only critique was to change the background sense color so it would stand out from the conversation.

MacBook Pro – 5

Hi-Fidelity Protoype





The camera captures the live video of all people in the conversation and the application shows a waist-up view of all people on the top half of the mobile screen, with a larger image of the person talking. 

The live translation of the conversation overlays on the bottom half of the screen with the name of people.

The chlickable yellow notification banner within the conversation shows the background noise.

Lori responds to the conversation either through Deaf speech or by typing on a text box in the far bottom, which will be turned into speech for the audience.




When the background notification is clicked, it would open a new window that would show intervals of past noise from 15 minutes ago, 10 minutes ago and what was picked up moments ago.

Converse App

The Prototype



From this project, I have learned that the creation of a new product should be people and problem focused, and not featured focus. I had a valuable experience of learning from people with a range of perspective through co-design activities and it helped me to get closer to the final user of the product. Regarding the iterative process, I realized that the solution and design would be very defferent when it's done directly with the final user through co-design sessions. In addition, by working on inclusive design projects, we can shift our design thinking towards universal solutions. 



The biggest limitation of our design was the lack of sign language and deaf speech to text and audio. Considering that typing is a last resort for Deaf people to respond, in the near future when deaf speech to text technology is commercially viable, Lori will be able to speak back to the device instead of having to type.