Enhancing learning accessibility through fully automatic captioning

The simple act of listening or of taking notes while attendinga lesson may represent an insuperable burden for millions ofpeople with some form of disabilities (e.g., hearing impaired,dyslexic and ESL students). In this paper, we propose anarchitecture that aims at automatically creating captions forvideo lessons by exploiting advances in speech recognitiontechnologies. Our approach couples the usage of o-the-shelf ASR (Automatic Speech Recognition) software with anovel caption alignment mechanism that smartly introducesunique audio markups into the audio stream before givingit to the ASR and transforms the plain transcript producedby the ASR into a timecoded transcript.