Zoom Auto-Captioning Redesign
Accessible & Inclusive Design Project
Type: Group Research & Design Project
Date: March 2025
Role: Researcher & Designer
Overview
This project focused on redesigning Zoom’s auto-captioning system to better support Deaf and Hard-of-Hearing (HoH) users. While Zoom provides automatic captions, research and user surveys revealed significant limitations that reduced accessibility. Our goal was to understand real-world frustrations and create an improved captioning experience aligned with accessibility standards (WCAG) and inclusive design principles.
Problem Statement
Only 14% of surveyed users found auto-captions fully accessible.
Common issues included:
Delayed or inaccurate captions.
Illegible or non-customizable text size.
Lack of context and speaker identification.
Limited user control (keyboard/motor accessibility).
Process & Methods
Literature Review – Studied accessibility challenges in AI-driven captioning tools and the role of captioning as both an assistive and instructional technology.
Survey Research – Distributed a survey to Deaf/HoH users via Reddit and NTID networks (10 participants).
Collected feedback on frustrations, coping strategies, and needs.
Example user strategies included lip-reading, asking speakers to repeat, or relying on interpreters.
Accessibility Frameworks – Applied WCAG guidelines and drew on feminist HCI methods to ensure user participation and minimize design bias.
Collaborative Design – Brainstormed improvements in visual, motor, and cognitive accessibility categories.
Key Findings
Feature Frustrations: Users cited noisy environments, caption delays, and inaccurate word labeling as major barriers.
Accessibility Gaps: Lack of customization (e.g., text size) excluded many users.
Social Dimension: Accessibility isn’t just technical — it affects participation, confidence, and inclusion in conversations.
Proposed Redesign
Visual Improvements:
Minimum 18px body text for readability.
Speaker profile picture + name alongside captions.
Blur background effect to improve text clarity.
Motor Improvements:
Keyboard shortcuts for enabling/disabling captions.
Tabbing support for transcript navigation.
Cognitive Improvements:
Clear sequencing of captions for better comprehension.
Contextual transcripts that align actions with dialogue.
Ability to reply directly to live-captioned comments.
Impact & Reflection
This project deepened my understanding of how technical, social, and ethical dimensions intersect in accessibility design. It also gave me practical experience in user research, inclusive prototyping, and applying accessibility standards to real-world platforms like Zoom.
Acknowledgment
I would like to thank Dr. Deborah Garwood for her guidance and support throughout this project, and my teammates Courtney Chan, Parker Hartman, Jay'llen Hathman, Caitland Okorafor, and Preeti Shenoy for their collaboration and creativity. I am especially grateful to the Deaf and Hard-of-Hearing participants who shared their experiences with us — their insights were invaluable in shaping a more inclusive redesign.