ViMG: a contactless deep learning framework for estimating sEMG amplitude envelopes from facial videos

Traditional surface electromyography relies on contact electrodes that can restrict natural facial movements and cause discomfort. These limitations are particularly critical in cognitive and developmental research, where sensor placement can alter the very behavior being observed, especially in longitudinal monitoring. We present Video Myography (ViMG), an approach for the contactless estimation of window-averaged zygomaticus major sEMG envelope amplitude from facial video data. Rather than reconstructing the raw sEMG waveform, ViMG predicts a single scalar per 0.7-s video window, corresponding to the mean amplitude of the processed root-mean-square sEMG envelope over that window. Our method employs a hybrid 3D U-Net architecture integrated with a Long Short-Term Memory network intended to capture spatiotemporal dynamics of perioral facial movements. On a dataset of 23 subjects recorded in a controlled laboratory setting, we estimate window-averaged sEMG envelope amplitude over short temporal windows, achieving a mean Pearson correlation coefficient of 0.661 (range across the five folds: 0.536–0.872). A landmark-based baseline achieved comparable performance ( r = 0.670 ) , indicating that the recoverable signal is largely explained by coarse perioral kinematics and can be captured by substantially simpler models. Gradient-based saliency maps indicate that the model’s predictions are informed by anatomically plausible perioral regions. While currently limited to controlled environments, this work establishes a proof-of-concept for contactless physiological assessment and a starting point for future contactless monitoring systems.