Simultaneous multimodal fNIRS-EEG recordings reveal new insights in neural activity during motor execution, observation, and imagery

Motor execution, observation, and imagery are important skills used in motor learning and rehabilitation. The neural mechanisms underlying these cognitive-motor processes are still poorly understood. We used a simultaneous recording of functional near-infrared spectroscopy (fNIRS) and electroencephalogram (EEG) to elucidate the differences in neural activity across three conditions requiring these processes. Additionally, we used a new method called structured sparse multiset Canonical Correlation Analysis (ssmCCA) to fuse the fNIRS and EEG data and determine the brain regions of neural activity consistently detected by both modalities. Unimodal analyses revealed differentiated activation between conditions; however, the activated regions did not fully overlap across the two modalities (fNIRS: left angular gyrus, right supramarginal gyrus, as well as right superior and inferior parietal lobes; EEG: bilateral central, right frontal, and parietal). These discrepancies might be because fNIRS and EEG detect different signals. Using fused fNIRS-EEG data, we consistently found activation over the left inferior parietal lobe, superior marginal gyrus, and post-central gyrus during all three conditions, suggesting that our multimodal approach identifies a shared neural region associated with the Action Observation Network (AON). This study highlights the strengths of using the multimodal fNIRS-EEG fusion technique for studying AON. Neural researchers should consider using the multimodal approach to validate their findings.