An oculometrics-based biofeedback system to impede fatigue development during computer work: A proof-of-concept study

Figures

• 

  Fig 1. The task timeline in the manual and automatic sessions (see micro-breaks). A schematic view of the exercise and screen information during a micro-break, and the experiment set-up. [The individual in this manuscript has given written informed consent (as outlined in PLOS consent form) to publish these case details].

• 

  Fig 2. The receiver operating characteristics (ROC) curve with the False Positive Rate (FPR) on the x-axis and the True Positive Rate (TPR) on the y-axis, depicted for the training and test sets, as well as the confusion matrix for the DT Ensemble model. The ROC curve is illustrated by computing the TPR and FPR averaged across the participants for varying values of the posterior probability threshold in [0 1] for the training and test sets. The confusion matrix was computed for the chosen KSS threshold of 5.

• 

  Fig 3. The architecture of the biofeedback system including the approaches to develop the statistical model to detect fatigue, and to trigger the micro-breaks. This architecture provides the flowchart of the main steps to develop the fatigue detection model, from the feature selection to the model evaluation and the deployment of the DT-Ensemble model in the biofeedback framework wherein the data were streaming from the eye tracker in real-time and the selected features were extracted in the end of each task segment and were fed into the deployed DT-Ensemble model to trigger the micro-break if fatigue was detected in the automatic sessions (light blue path), whereas the triggering of the micro-break was only based on the decision of the participant in the manual sessions (red path).

• 

  Fig 4. (a) Classification performance (ACC) of the DT Ensemble model for the male and female participants in the manual (without BF) and automatic (with BF) sessions, where BF stands for Biofeedback, (b) The ACC, Mean ± SD, in different time of the day (Morning and Afternoon).

• 

  Fig 5. Comparison of the overall performance (OP) across the automatic (with biofeedback) and manual (without biofeedback) sessions to assess the effect of micro-breaks, with the indicated segments with significant difference in the OP (p < 0.05) (a), and across the first and second sessions to inspect whether there is a learning effect (b). The points and error bars respectively represent the mean and standard deviation values across the participants for each segment.

• 

  Fig 6. The weighted scores of the NASA-TLX subscales in the automatic sessions (with biofeedback) and manual sessions (without biofeedback), where the subscales range from 0 to 33 indicating low to high levels. The subscales are Mental Demand (MD), Temporal Demand (TD), Performance (PF), Effort (EF), Frustration (FR), and Physical Demand (PD).

• 

  Fig 7. The obtained ratings of total task load index (TLX) and NASA-TLX subscales, i.e. Mental Demand (MD), Physical Demand (PD), Temporal Demand (TD), Performance (PF), Effort (EF), and Frustration (FR). The participants are separated by their sex on the x-axis to males (1–10) and females (11–20). The NASA-TLX scores are depicted separately for the automatic and manual sessions.

• 

  Fig 8. Subjective ratings of fatigue (KSS scores) in the automatic (with biofeedback) and manual (without biofeedback) tasks. The segments with significantly different KSS scores are indicated by the red color for the manual sessions and black color for the automatic sessions (p < 0.05). The points and error bars respectively represent the mean and standard deviation values across the participants for each segment.

• 

  Fig 9. A representation of the overall performance (OP) of each participant (Y-axis) in the manual and automatic sessions. The presence and absence of micro-breaks are indicated respectively by “1” and “0” at the end of each segment (X-axis). The OP is color coded with the color bar indicated on the right side of the graph with blue for lower and green for higher task performance.

• 

  Fig 10. The changes through TOT in the oculometrics, i.e. Blink Frequency (BF), Percentage of the duration of closed eyes to opened eyes (PERCLOS), Saccade Peak Velocity Amplitude Relationship (SVA), Saccade Frequency (SF), Pupil Diameter Interquartile Range (PDIR) used in the deployed model in the automatic (with biofeedback) sessions and manual (without biofeedback) sessions. The points and error bars respectively represent the mean and standard deviation values across the participants for each segment. The segments with significant differences according to the pairwise comparisons are marked by “*”, (p < 0.05).