Tracking of Mental Workload with a Mobile EEG Sensor

Figures

• 

  Figure 1. Study protocol. Participants completed two lab visits. On day 1 they received a diagnostic screening and neuropsychological assessment, and had to fill in the pre-assessment of the usability questionnaire. On day 1 and day 2, participants completed the PANAS to assess the affective state prior to and after the EEG recordings. Prior to the start of the cognitive tasks, participants received standardized instructions for the setup of the mobile EEG device. After successful application, a fixation cross was displayed and participants were instructed to keep eyes open, followed by two minutes of rest with eyes closed. Afterwards, participants completed the N-back task with three difficulty levels (presented in randomized order) and the Go/No-go task. On day 2 only, participants filled in the TUI-II-post questionnaire and the System Usability Scale after completion of the tasks.

• 

  Figure 2. (A) Electrode placement. The figure depicts electrodes that are covered by the Emotiv Epoc mobile EEG device. Frontal electrodes are depicted in blue (n = 8), temporal electrodes in green (n = 2), parietal electrodes in yellow (n = 2) and occipital electrodes in red (n = 2). (B) Mobile setup. Display of the mobile EEG device and the tablet that was used for stimulus presentation. The figure illustrates an example of the N-back task. Participants were instructed to press the ‘button’ (visible on the lower part of the tablet screen) upon detection of a target stimulus. The rectangular button covered an area of 172 mm × 20 mm.

• 

  Figure 3. Illustration of the three difficulty levels of the N-back task. Participants were instructed to press a button as fast as possible whenever a target appeared. Each condition contained 150 stimuli (50 targets) that appeared on the screen for 1600 ms each. (A) Participants were instructed to press a button whenever the digit ‘1’ appeared on the screen. (B) Participants had to press a button when three digits in a row were even. (C) Participants were asked to press a button when the digit was identical to the second to last digit.

• 

  Figure 4. The continuous EEG data were preprocessed (band pass filtered at 1–40 Hz, channel rejection) and split into event-related epochs of 1500 ms each. After epoch rejection, powers of the individual frequency bands (delta-beta) were extracted.

• 

  Figure 5. After preprocessing, the EEG data vector (input) has length 56 (4 bands × 14 channels). Correspondingly, the number of ANN input nodes is 56, the output layer during the optimization process has 2 nodes (we classify only L0 and L3 difficulty levels). The number of hidden layers vary between 1 and 3, and the number of hidden nodes is kept the same for all hidden layers and takes values 1, 5, 10, 15, 20, 30, 40, 50, 60, 70, 80, 90.

• 

  Figure 6. The behavioral analysis confirmed the experimental conditions as reflected by increasing task demand with slower responses and reduced accuracy across the three conditions (corresponding to difficulty levels L1, L2, L3). (A) Mean reaction times across days for each condition. A significant main effect of condition emerged. Post hoc tests indicate a significant difference between L1 and L2 and L1 and L3. *** p = 0.001. (B) Mean fraction of correct responses across days for each condition. A significant main effect of condition emerged. Post hoc tests indicate a significant difference between all conditions. *** p = 0.001, ** p = 0.005.

• 

  Figure 7. Marginal means with confidence intervals in three-factor ANOVA of power measures. The panels illustrate the logarithmically scaled power measure in the different EEG frequency bands (A): theta, (B): delta, (C): alpha, (D): beta. On the x-axis, the different conditions are mapped with increasing cognitive load (L0: baseline; L1: detection; L2: one-back; L3: 2-back). Data are averaged for electrodes over the four cerebral lobes (red: frontal; yellow: temporal; cyan: occipital; orange: parietal). Stars in the same color highlight significant post hoc comparisons across conditions for the respective lobe (e.g., a blue star on the line between L1 and L2 indicates a significant difference between these conditions for the frontal lobe with * p < 0.01). The robust replication of the primary hypotheses using the mobile technology can be observed as: increase of frontal theta (red curve and stars in Panel A) and occipital alpha (blue in Panel C) with increasing cognitive load (from L0 to L4) decreased with higher cognitive load. Further, a similar pattern emerges for the delta band and potentially the beta band as well. * p < 0.01, a.u.: arbitrary unit.

• 

  Figure 8. Accuracy of the ANN classifier for different train/test data divisions (training data from 1.5 day, training data from 1 day, training and validation data from separate subjects). The accuracy is shown for the separation of all four conditions (25% chance level, orange bar) and between each pair of conditions (50%). The classification accuracy profits from using data from the same day (1.5 day condition) and from the same subject (1 day condition) as well as from a stronger difference between the cognitive load conditions (e.g., L0–L3 versus L1–L2). This pattern suggests that inter- and intra-subject variance affect the performance but also that the network seem to rely on relevant cues since the separation is improving with the load difference.

• 

  Figure 9. ROC (receiver operating characteristic) curves for two-classes discrimination for the train (blue) and test (red) data. AUC (area under the curve) is correspondingly 0.98 and 0.92 for the train and test data.

• 

  Figure 10. Confusion matrix for four conditions discrimination (classes 0, 1, 2, 3). Overall, an accuracy of 48.4% almost doubles the random guess baseline (25%) and the confusion between neighboring classes is higher than between more distinct ones.

• 

  Figure 11. The results of cross validation with a leave-one-out approach. The classifier separates L0 and L3 in the 1.5 days of training data condition. Subject 2 was removed during the initial data preparation stage due to insufficient data quality. The average score was 81.1% and all subjects achieved accuracy above chance level.