Cross-Subject EEG-Based Emotion Recognition Through Neural Networks With Stratified Normalization

Figures

• 

  Figure 1. Our experimental setup captures the effects that the experimental data, labeling method, feature extraction method, and normalization method have onto the emotion recognition accuracy of the models.

• 

  Figure 2. Architecture of the classifiers. Features first go through a min-max normalization of the data before being input to the neural network. The first three layers consist of dense layers with 64 neurons, a dropout for the first two, ReLu activations, and either a batch or a stratified normalization. The last layer is a 3-neuron dense layer that outputs the classification prediction through a Softmax function.

• 

  Figure 3. Batch normalization method. The data is normalized per feature, independently of the participant, and session.

• 

  Figure 4. Stratified normalization method. The data is normalized per feature, participant, and session.

• 

  Figure 5. The boxplots of the figure indicate the distribution of the emotion recognition accuracies (y-axis) of the models in each experimental condition, with either (1) a binary classification (left graph) or ternary classification (right graph) labeling method, (2) a Welch, multitaper, or Differential Entropy (DE) feature extraction method (graph's columns), (3) a batch (blue bars) or stratified (orange bars) normalization method. The value inside the boxplots is the median value of the distribution. Besides, the figure also reports the p-value of the dependent-samples t-test between the two normalization method conditions for each feature extraction method condition.

• 

  Figure 6. This figure indicates the classification results for the emotion recognition and participant identification in the input, hidden layers, and output of the neural network using multitaper as feature extraction method. (A,B) Plot the emotion recognition accuracies, and (C,D) show the participant identification accuracy.

• 

  Figure 7. Embedding of the input of the neural network with batch normalization and with three emotion categories.

• 

  Figure 8. Embedding of the input of the neural network with stratified normalization and with three emotion categories.

• 

  Figure 9. Embedding of the output of the neural network with batch normalization and with three emotion categories.

• 

  Figure 10. Embedding of the output of the neural network with stratified normalization and with three emotion categories.