Closed-Loop Brain Machine Interface System for In-Vehicle Function Controls Using Head-Up Display and Deep Learning Algorithm

Human-vehicle interfaces have evolved with technologies such as touchscreens, voice commands, and advanced steering wheels with control panels. However, these technologies can increase the driver’s cognitive workload and distraction, potentially compromising road safety. Brain-machine interfaces (BMI) offer an alternative way of controlling in-vehicle features with minimal distraction. This study presents a closed-loop steady-state visual evoked potentials (SSVEP) based BMI for controlling in-vehicle features via a windshield head-up display to control multiple vehicle features such as music, temperature, settings, and navigation system. The custom software synchronizes visual stimuli, processes electroencephalogram (EEG) signals, and selects target icons in real-time. A convolutional neural network (CNN) based on the SE-ResNet architecture efficiently detects SSVEPs and the driver’s intended target icon in under 1 second. Comparative evaluations against traditional and deep learning methods using time and frequency features show superior performance. With just 225 seconds of calibration data, the proposed compact deep-learning model enables drivers to adjust their environment within 0.5 seconds, achieving an Information Transfer Rate (ITR) of 93.43 <tex-math notation="LaTeX">±</tex-math> 7.26 bits/min. This SSVEP-based BMI system demonstrates multi-layer in-vehicle feature control with ten human participants, highlighting its potential to enhance road safety by allowing drivers to make adjustments without diverting their attention from the road. Our code and dataset can be downloaded from https://github.com/hosseinhamidi92.