Description
Recent advancements in the accuracy of Automated Speech Recognition (ASR) technologies have made them a potential candidate for the task of captioning. However, the presence of errors in the output may present challenges in their use in a fully automatic system. In this research, we are looking more closely into the impact of different inaccurate transcriptions from the ASR system on the understandability of captions for Deaf or Hard-of-Hearing (DHH) individuals. Through a user study with 30 DHH users, we studied the effect of the presence of an error in a text on its understandability for DHH users. We also investigated different prediction models to capture this relation accurately. Among other models, our random forest based model provided the best mean accuracy of 62.04% on the task. Further, we plan to improve this model with more data and use it to advance our investigation on ASR technologies to improve ASR based captioning for DHH users.
Date of creation, presentation, or exhibit
9-2016
Document Type
Conference Paper
Department, Program, or Center
Information Sciences and Technologies (GCCIS)
Recommended Citation
S. Kafle, M. Huenerfauth, "Effect of Speech Recognition Errors on Text Understandability for People who are Deaf or Hard of Hearing," in 7th Workshop on Speech and Language Processing for Assistive Technologies, INTERSPEECH, 2016.
Campus
RIT – Main Campus
Comments
Copyright 2016 INTERSPEECH. Presented at the 7th Workshop on Speech and Language Processing for Assistive Technologies, INTERSPEECH, September 8-12, 2016, San Francisco, California.