By: Soumya Avva, Alex Thomas, Debbie Yuen

Approximately 10 years ago, my family got our first iPad. My sister’s therapists and high school teachers had recommended augmentative and alternative communication (AAC) applications on iPads. Excited to see how iPads worked, I watched my parents and my sister’s therapists download AAC apps and customize the settings. Impressed with the iPad’s functionality, I wanted to learn how to create mobile apps and design something aesthetically pleasing like the iPad. As I grew older, I saw that my sister and other users with disabilities experienced limitations in accessing the iPad and navigating mobile applications designed for them. Motivated to be able to design something of my own one day, I learned how to program. I attended events at the Ed Roberts Campus and joined the organization, Best Buddies. I explored ideas of inclusion in my independent research and senior thesis paper. Learning from and discovering the needs individuals with disabilities have, I was inspired to create Euphemia.

Problem and Motivation

Communication can be especially challenging for people with developmental disabilities. People without communication disabilities may rely on technologies including smart phones, e-mail, and video chat to connect with others on a daily basis. However, many of these technologies aren’t always accessible to the disability community. There is a great need to continue improving communication technology to bring more accessible communication tools to people with communication disabilities. For individuals unable to speak, write, and read, current AAC tools are helpful when communicating with others who are physically present. People with communication disorders need the tools to be able to speak for themselves independently, especially during times when their health needs attention. This is especially the case when it comes to the virtual world and remote communication. Without tools to communicate remotely, people with developmental disabilities may experience difficulty voicing their needs and feelings. There is also a need to expand current AAC systems to bridge the gap between those who are verbal and those who are nonverbal.

How might we change how people with developmental disabilities experience communication especially during times when remote communication is necessary?

Promoting accessible communication during the COVID-19 pandemic is essential. Due to COVID-19 and social distancing, communication has become increasingly challenging especially for those with disabilities. Many traditional AAC devices and non-devices require physical presence. This makes it significantly more difficult for people with disabilities who are nonverbal to participate in activities on Zoom, connect with friends and family far away, and remotely communicate. Without the necessary tools, it can be difficult to express and determine whether they are in need of medical attention.

“In many cases they can’t speak for themselves, or have anxiety issues, with nobody there who they know who can support them,” - James Moran. (New York Times).

COVID-19 is hitting long-term care facilities and group homes for the disabled hard. Some are unable to communicate their emotions and needs and/or lack the resources to communicate with others. People who care for those with disabilities, or their support team, are also facing many challenges amid the pandemic (ABC News). Parents and caregivers are experiencing difficulty communicating with their child or client about what is going on in the world and are dealing with increased levels of stress. Those part of an individual's care team needs support, especially during a time that can significantly affect mental health. People with little or no communication abilities need a way to be part of and connect to the world.


Persona: A persona on Kate, an individual with developmental disabilities.

Euphemia is an AAC mobile and web application designed to bridge the nonverbal and the verbal to encompass different forms of conversations. We support multi-user system behavior with our two account options, square and circle . A user is defined as a different person, each with their own customizable account and user profile.

Square Accounts: Kate is an example of someone who may find square accounts beneficial.

Square accounts are for users who benefit from using different forms of communication. For example, Picture Exchange Communication System (PECS), American Sign Language (ASL), gestures, and sound cues. These accounts are designed to have simple user interface and user experience. They do not require advanced actions and are not overloaded with information and sensory details.

Circle Accounts: Kate's therapist, a member of the support team, is able to plan out schedules ahead of time.

Circle accounts is an extension of square accounts. These accounts can optionally link to multiple square accounts and act as an administrator by allowing information and customizing user square account settings. Circle accounts may allow access control policies to other circle /square accounts. For example, a therapist with a circle account working directly with a square account user may want to add PECS icons or modify their daily schedule.

Admin Circle Accounts: Kate's parents are examples of individuals who may have administrative accounts.

Administrative circle accounts may have access to information to the square account and may customize user settings for the Square acount. For example, a parent or legal guardian may have an administrative circle account to see personal health information, location, and user statistics.

The Network: Shows how everyone is able to remain connected with each other while maintaining privacy.

Euphemia’s user account system to allow access control policies between circle and square users ensures privacy preservation. For example, friends of the nonverbal user do not necessarily require access to information such as behavior statistics or personal health information and still want to communicate with the nonverbal user. The account system will limit the access for some users that may communicate with the nonverbal user.

Currently, Euphemia is designed for people with developmental disabilities whom may be nonverbal, unable to write and read, and unable to use sign language. Euphemia aims to support users with a range of fine-motor and communication skills as the project continues to move forward. Its customizable modes accommodates users with a range of different needs.

Most research participants were people who have been diagnosed with profound/severe autism and or are people from their support team. However, I am expecting to interview people with different needs in the future.

These design requirements were based on my research findings. I believe that it is essential that I keep these requirements in mind while designing in order to successfully meet user needs.

1 / 3
2 / 3
3 / 3
3 / 3
3 / 3
3 / 3
3 / 3
3 / 3
3 / 3
3 / 3

A storyboard that illustrates how Euphemia is used.

How to Use: This is how Kate and her mother can communicate

How to Use: This is how Kate can join Zoom calls quickly

How to Use: This is how Kate can communicate with her friends

PECS Keyboard: This illustrates how a parent can text their child to ask them a question using the PECS keyboard. They may send a sequence with the PECS keyboard. Words written in English (or some other language) will be translated to PECS when the client/child views the message.

Schedule: The user’s schedule. The picture of what they are currently on shows up at the top. Clicking on the icon will read out loud the title, "nap time". The water tank shows the user where they are in their schedule. Swiping and down shows the rest of the schedule. Swiping right or left takes you to the call/facetime or PECS icon book pages.

PECS Icon Book: This screen shows all of the user’s PECS icons. Each folder will be organized by most used, location, and time. Instead of sorting through all the folders and images, users may also use sign language and sound to find the image they are looking for. Users may also easily join Zoom meetings or other virtual events by tapping on the image.

Call and Video Chat: Double tap on the top section to video chat. Swipe left or right to end.

Camera Feature: The yellow square shows that an image is detected. If there is no orange in the PECS icon book, the user may add it as a new icon. If there is an orange in the PECS icon book, the book will find it for the user.

Programming: Developing the PECS screens in Xcode.

Programming: An early example of the PECS screens.

Future Steps

In collaboration with peers, I am currently working on developing Euphemia. Currently, I am contributing to programming the PECS user interface and creating graphics for PECS icons. In the future, I hope to further investigate different avenues people with disabilities can use to communicate (i.e. example, gestures, sounds, voices, and movement). Integrating machine learning models will allow people with communication disabilities to experience communication differently. After intensive user testing and research, I would like to publish Euphemia as a free application for users. I envision Euphemia to be distributed to as many different platforms as possible to benefit the most amount of people.

My Role