Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in Games

Figures

• 

  Figure 1. General structure of our proposed method for remote detection of stress and boredom of players during their interaction with games.

• 

  Figure 2. Calibration phase composed of emotion elicitation games (calibration games) and remote acquisition of signals from the user. The result of this phase is a user-tailored model applied to detect emotions.

• 

  Figure 3. Emotion estimation phase. Remotely acquired signals from the player are fed into a user-tailored model that outputs the stress/boredom levels of the player during the interaction with an ordinary game.

• 

  Figure 4. Structure of a calibration game. The x-axis shows the progression of time. The y-axis shows the game difficulty and the emotional state users should experience, i.e., stress and boredom.

• 

  Figure 5. Screenshots of the three calibration games used in the experiments. From left to right: Mushroom, Platformer, and Tetris. Reproduced from Bevilacqua et al. [23].

• 

  Figure 6. Experiment setup. On the left, an image illustrating the position of the equipment, including the angle of the external light source; on the right, an image highlighting the position and angle of the video camera.

• 

  Figure 7. Two (uninterrupted) parts of the experiment. (a) calibration part; (b) testing part. G: calibration game, Q: questionnaire about game/level, R: resting, ABC: levels of Infinite Mario, E: demographic questionnaire.

• 

  Figure 8. Screenshots from Infinite Mario. From left to right, level types Overground, Underground and Castle, respectively.

• 

  Figure 9. Extraction of video segments H0 and H1 containing boring and stressful interactions, respectively, in the calibration games. Initial D seconds of any video Ci,g are ignored and the remainder is divided into three segments, from which the first and the last ones are selected. Stripes highlight discarded video segments.

• 

  Figure 10. Iteration of a 3-fold Leave-One-Session-Out Cross-Validation performed on the gaming session of a given subject with three games, i.e., A, B and C. Data of two calibration games, e.g., A and B, are used to train the machine learning model, while data of the third calibration game, e.g., C, are used to evaluate the model.

• 

  Figure 11. Training and evaluation of a user-tailored emotion classifier. (a) training of the emotion classifier; (b) construction of a testing dataset; (c) evaluation of the emotion classifier.

• 

  Figure 12. Distribution of accuracy values at a subject level. (a) histogram showing the number of subjects and the accuracy rate obtained in their emotion classification; (b) density curve regarding the distribution of accuracy values.