Multimodal Hybrid Deep Learning for the Detection of Autism Spectrum Disorder (ASD) Using Eye-Tracking Data

Background: Autism spectrum disorder (ASD) assessment may benefit from objective measures of visual attention, although eye-tracking classifiers remain research tools rather than replacements for clinical diagnosis. Objective: This study compared eye-tracking representations and sequence encoders for binary classification of ASD versus typical development using the Saliency4ASD dataset. Methods: Eye-movement sequences, fixation maps, stimulus images, and tabular features combining gaze and stimulus content were investigated in a hybrid learning pipeline that combined sequence encoders, ResNet-18 image-processing branches, and a multilayer perceptron (MLP) through feature concatenation. The sequential experiments evaluated six data-partition ratios, 15 modality combinations, seven optimization configurations, and alternative sequence architectures. The selected model was also evaluated across five additional holdout partitions grouped by stimulus image. Results: The combination of scanpath and fixation-map representations achieved a file-level F1 score of 0.866 during modality confirmation. The long short-term memory (LSTM) + gated recurrent unit (GRU) configuration achieved a file-level F1 score of 0.9333 and a receiver operating characteristic area under the curve (ROC-AUC) of 0.9733 on the originally selected test partition comprising 30 stimulus images. Five additional image-grouped partitions yielded a mean file-level F1 score of 0.839 ± 0.015. The random forest baseline achieved an F1 score of 0.8999 on the original partition; however, it was not evaluated across the additional partitions. Conclusion: The results demonstrate the potential of combining temporal and spatial gaze representations while highlighting sensitivity to data-partition selection. Reuse of test data during model tuning, limited subject independence, and the absence of verified generalization to previously unseen children preclude claims of independently validated diagnostic performance.