Helping students keep up with real-time captions by pausing and highlighting

We explore methods for improving the readability of real- time captions by allowing users to more easily switch their gaze between multiple visual information sources. Real-time captioning provides deaf and hard of hearing (DHH) users with access to spoken content during live events, and the web has allowed these services to be provided via remotely- located captioning services, and for web content itself. However, despite caption benefits, spoken language reading rates often result in DHH users falling behind spoken content, especially when the audio is paired with visual references. This is particularly true in classroom settings, where multi-modal content is the norm, and captions are often poorly positioned in the room, relative to speakers. Additionally, this accommodation can benefit other students who face temporary or "situational" disabilities such as listening to unfamiliar speech accents, or if a student is in a location with poor acoustics.