Automated Measurement of Horizontal Strabismus in Children’s Primary Gaze Photographs Using Deep Learning and Computer Vision

• This study proposed an AI system using deep learning and computer vision techniques, which produced predictions similar to the measurement by Hirschberg test. • The deep learning algorithms were trained on an extensive dataset with nearly ten thousand gaze photographs from different populations. Consequently, our systems achieved high levels of accuracy comparable to previously DL or CV systems. • These results are encouraging proof for automated screening, and the system is reliable and accurate to measure horizontal strabismus in gaze photographs. To develop and validate algorithms that automatically measure horizontal ocular alignment in children’s primary gaze photographs using deep learning and computer vision. We proposed two-stage artificial intelligence (AI) system, including deep learning (DL) algorithms for automatic segmentation of eye structure and computer vision (CV) algorithms to measure horizontal ocular alignment. DL algorithms were trained using a public ocular images dataset. The measurements of CV algorithms were tested in primary gaze photographs recruited from a tertiary hospital. The DL training dataset involved 11,018 ocular and corresponding ground truth images. We then used 147 primary gaze photographs to validate the CV algorithms. The strabismus angles measured by the AI system closely followed the Hirschberg test (HT) using linear regression analysis (slope = 0.919, p < 0.001). There was excellent agreement between the two methods, as measured by the intra-class correlation coefficient (ICC = 0.98). Compared to the angle measured by HT, the proposed algorithms achieved a limits of agreement (LoA) of -7.1 to 6.3 prism diopters (PD) and coefficients of variation (CoV) of 13.5%. In conclusion, we proposed an automated DL and CV system to measure horizontal ocular alignment using gaze photographs. Our system produced predictions similar to the measurement by Hirschberg test. In addition, the two-stage AI system provided an explanation of the rationale for clinical decisions.