Automated Token-Level AOIs Using OCR for Predicting Novice Programmers’ Visual Attention

This paper introduces Gaze2Code, a novel cognitive modeling framework designed to support novice programming comprehension through automated token-level Area of Interest (AOI) generation using Optical Character Recognition (OCR). By aligning OCR-processed code images with eye-tracking data, the framework achieves precise fixation-to-token mapping, enabling fine-grained analysis of visual attention. A robust OCR optimization pipeline—incorporating adaptive thresholding, contrast normalization, and token confidence filtering—significantly improves token detection accuracy, bounding box alignment, and semantic coverage. Empirical evaluations across multiple programming tasks demonstrate substantial gains in AOI quality and predictive performance, with optimized AOIs enhancing BiLSTM model fit in modeling gaze behavior. The framework not only advances high-resolution attention modeling but also enables the concrete identification of comprehension strategies, such as novice regressions to operators or repeated fixations on delimiters and loop constructs, thereby supporting the development of adaptive, gaze-sensitive learning environments. This work represents an important step toward scalable and intelligent tutoring systems that deliver real-time feedback in programming education.