Enhancing EEG Decoding with Selective Augmentation Integration

Figure 1. A typical auditory decoding process that extracts meaningful auditory/visual stimulus information from EEG signals, aiding in the exploration of the neural mechanisms underlying speech/vision perception.

Figure 2. End-to-end enhancing EEG Siamese network framework with selective augmentation integration. In the beginning, raw EEG data is split into two parts for enhancing training and simple training with the Siamese network. Then, we introduce a cross-attention module and contrast learning to align enhanced embedding and original embedding by learning their similarity. Finally, we adaptively select a better integration via a trainable coefficient and then calculate Pearson correlation values of pairs of auditory EEG data.

Figure 3. Visual exemplars pertaining to the augmentation techniques. (a) An example of original test EEG data, (b–h) The EEG data after the corresponding augmentation techniques are applied to the original EEG data.

Figure 4. Illustration of NeuroBrain model.

Figure 5. Details of Pearson correlation scores for different subjects on SparrKULee. The colored lines and red points in the boxes represent the models’ median scores and average scores, respectively.

Figure 6. The loss for the train set and Pearson coefficient for the validation set through training steps. (a) Training loss; (b) validation Pearson coefficient.

Figure 7. Reconstructed mel-spectrogram from the same EEG signal and the ground-truth speech stimuli. (a) Linear; (b) VLAAI; (c) FastSpeech2; (d) HappyQuokka; (e) NeuroBrain; (f) Ground-truth.