Toward stealth assessment of reading comprehension

Reading comprehension is a critical skill that enables us to use text for learning, work, and pleasure. It is typically assessed through multiple choice or open-ended questions based on the text. In this article, we explore a novel machine-learning approach to assessing comprehension based on the properties of the text and of its oral reading that are known to be connected to comprehension. This method of assessing comprehension can potentially function as stealth assessment, running continuously and unobtrusively in the background while students read aloud during, for example, supplemental independent reading. Furthermore, we estimate student comprehension based on a partial oral reading of a text. This design allows the system to act on the results of the stealth assessment in real time, immediately following the reading, by providing, for example, an overt comprehension check and feedback when lack of comprehension is predicted based on the analysis of oral reading. Our best models show promising results, with over 80% precision in predicting successful comprehension of a passage read orally.