IGM-MELv2: Infrared Guiding Modal Multiuser Eye Localization System on ARM CPU for Autostereoscopic Displays

Binocular autostereoscopic display requires a real-time eye localization system with high accuracy and robustness. However, despite a decade of development in deep learning, few models are suitable for real-time operation on an embedded device that can be assembled into the display. In this work, we propose a system-level design for real-time eye localization on a single ARM CPU, named Infrared Guiding Modal Multiuser Eye Localization v2 (IGM-MELv2). Aiming for effective solutions within a constrained computational budget, IGM-MELv2 deconstructs the complex multiuser eye localization challenge into two manageable tasks: thermal-infrared (TIR) face detection and face-region-based visible light eye localization. For TIR face detection, we search for a suitable accuracy-latency tradeoff and propose a tiny-lightweight detector named Yolo-Fastest-TF. For visible light eye localization, we transform a general object detector through a series of design choices, resulting in the tiny-lightweight Yolo-Fastest-VE. Additionally, an RGB-thermal Multiuser Low-resolution Eye Localization (RGBT-MLEL) dataset is collected to build the system. Experimental results reveal that IGM-MELv2 has a frame rate of 44.86 frames per second on a Raspberry Pi 4B and can successfully locate 97.4% of the eyes in the RGBT-MLEL test set. On the two separate tasks, the tiny-lightweight networks have accuracy and robustness comparable to complex models while maintaining high computational efficiency. Yolo-Fastest-TF achieves 91.46 AP50 on TIR face detection with 21MFLOPs computation. Yolo-Fastest-VE achieves a success rate of 100% under an error threshold of 0.15 on BioID with 37MFLOPs computation.