Inclusive Design, Product Design
Converse is an application that helps deaf people to communicate with others at work or personal life in a way that creates deep and meaningful connections.
Client: Student Project
My Role: Inclusive Designer, User Researcher, Prototyper
Tools: Sketch, Invision, Photoshop, Pen & Paper
Methods: Participatory Design, Storyboarding, Sketching, Prototyping, User Testing
Lori Leal was born deaf and learned sign language at 14 years old. She received family support to strengthen her comprehension and vocabulary however, she still struggled with daily communication. The barrier to communicate with hearing people at work and in her personal life prevents her from forming deep and meaningful relationships in which she feels isolated and alone.
Our participatory design activity was divided into five phases where we introduced design dimensions, technologies, and ideas available for use. In each phase, Lori picked aspects of the design she wanted to pursue. This paced the introduction of new factors so she could consider them individually before integrating them into a larger whole. It also built ownership in the design solution since she had picked all of them. We think this part of our process was very robust and had the intended effect.
PROBLEM UNDERSTANDING INTERVIEW
The team conducted an interview and brainstorming session with Lori to define a design problem or opportunity. Our interview session consisted of covering the basics of participatory design; the goal, the scope of the project, personal introductions and the structure of the interview. Then we asked Lori questions about her daily routine, work, family, her hobbies and what she did to enjoy herself. We focused on identifying specific points where there were unmet needs or problems without a solution that could be solved to improve the quality of her life and asked follow up questions to gain more insight into why.
Lori mentioned two interrelated problems: a lack of deep meaningful communication, which resulted in feelings of isolation and loneliness. We briefly reviewed the technology she used to currently overcome this problem and then talked more about the specific challenges to communication that prevented her from forming communication.
Our project focuses on alleviating the participant’s feelings by promoting deep meaningful communication. The specific findings from our interview with Lori that led to this focus are as follows:
1- Challenges in conversing and connecting with others
● Lori’s family does not sign and is the only deaf person in her family.
● She is trained to lip-read, however, she usually grasps an eighth of all words spoken.
● Lori avoids communication with hearing people as it is a frustrating experience.
● She is willing to ask people to write but it is often problematic or not possible.
2- Feeling isolated
● Lori worked at her previous job for 18 years but had no direct communication with anyone due to the communication barrier and not being able to hear them.
● She feels that using an interpreter for communication does not create a connection between them and another person since the interpreter is viewed as a proxy.
● While traveling solo, Lori noticed that people try to talk to her but then shy away when they realize that she is deaf.
● Even if having a conversation, it remains surface-level and not deep.
PROBLEM TO BE ADDRESSED
Based on the problems we identified in our first interview, it is clear that Lori’s biggest daily challenge is communication. The barrier to communication with hearing people at work and in her personal life prevents from forming deep meaningful relationships without which they feel isolated and alone. Therefore, in this project, we will focus on creating solutions that help her communicate with others in a way that creates the deep and meaningful connections that Lori seeks.
DATA ANALYSIS & IDEATION
During the data analysis process, four key tasks were defined that our participant may want to complete with the technology we were designing together. The tasks are independent of any specific technology so that we could still explore many designs. Through sketching, we brainstormed a range of design ideas for how to support those tasks and considered a mix of different platforms, like physical interactive objects, smartphones, tablets, and laptops may be helpful.
1. Converse independently: a) Talk to someone without an interpreter b) the Deaf person wants to respond to a conversation (family gathering, conversation at the workplace), or make quick comments on a fast-paced group conversation.
2. Invite others to or act on opportunities to converse: a) Invite or act on opportunities to converse with new people b) Responding to situational conversation (on the street, or shopping, or while hiking) while ordering for a simple need and improve the ability of communication (ordering food, ordering drinks …)
3. Having a deep one-on-one or group conversation: a) Have a nuanced (adopting a position on a topic, expressing an opinion, etc.) conversation discussion with a co-worker or family member. b) Deaf person wants to respond to a conversation (family gathering or meeting), or make quick comments on a fast-paced group conversation
4. General conversation skills: like social confidence, comprehension, etc.
CO-DESIGN SESSION & INSIGHTS
PRE CO-DESIGN PROTOCOL TEST OUT
We tested out the protocol we made with some pilot participants to make sure specific points, questions, and materials are working and modify the errors.
BUILDING INTERACTIVE PROTOTYPE
Considering Lori’s insights from the co-design session, we built a device including a 360° camera and an accompanying app called Converse, which would provide meaningful communication within desired social environments. The camera captures the live video of all people in the conversation and the application shows a waist-up view of all people on the top half of the mobile screen, with a larger image of the person talking.
Interaction Design & Visual Design Iterations
The live translation of the conversation overlays on the bottom half of the screen with the name of people. Lori responds to the conversation either through Deaf speech or by typing on a text box in the far bottom, which will be turned into speech for the audience. In the future, when deaf speech to text technology is commercially viable, Lori will be able to speak back to the device instead of having to type. This created our project focus to encourage inclusiveness and reduce isolation within a social environment.
We began the session by demonstrating some of the major changes and critiques from our co-design session to spark comments and insights. Then, we observed Lori while interacting with the prototype and asked questions to understand her satisfaction from the design concepts within the app and how it would function if it was used in a real-life situation.
Final participant comments and insights
One of the major improvements was the app’s ability to detect and display background noise. Lori stressed that without the ability to hear, there is a lack of access to the noise that is occurring around them. The background noise detection within the app would provide an understanding and an awareness that is lacking for a Deaf person within a hearing environment. She also appreciated the ability to change the sensitivity of the notification to alert any emergencies occurring around her.
We also provided a method of how the background noise could be notified, by a banner, within the conversation chat, or pop-up notification. Lori enjoyed seeing the notification embedded right into the chat and the only critique was to change the color so it would stand out from the conversation. She also appreciated that when the background notification was clicked, it would open a new window that would show intervals of past noise from 15 minutes ago, 10 minutes ago and what was picked up moments ago.
Lori loved the ability to customize the chat features such as changing the font size, change the chat speed, changing the color of the text, and changing the color of the background. For instance, someone who was colorblind was using the app, they would be able to access the chat by customizing it to their liking and ability. She really enjoyed that there was a microphone conveniently located on the main screen to utilize Deaf speech to text instead of typing. Lori also wanted the ability to control the conversation in her favor.
She wanted to be able to focus on certain people within the social environment. This would occur in moments of Lori using the app and desires to switch the large image to focus on someone else who she prefers to have a conversation with; this would give her some control over the device and who it focuses on. She also appreciated that we acknowledged that typing was a last resort.