Exploring indoor environmental perception through questionnaires, physiological monitoring, and facial expression recognition

Indoor environmental quality (IEQ) has a profound impact on occupant health, comfort, and performance. Various methods are employed to collect feedback from building users, ranging from conventional questionnaires, increasingly utilized physiological monitoring, to recently developed automatic facial expression recognition. This study explores the feasibility of detecting environmental perception and satisfaction by continuous monitoring of facial expression and physiological parameters and compares the results with questionnaire results. Using independently controlled twin climate chambers, we established two distinct air pollution and noise exposure conditions and invited 25 participants to experience them in counter-balanced orders. They engaged in different activities including resting, taking cognitive tests, and watching a relaxing landscape video. Survey results suggested stark differences in self-reported environmental perceptions between the two conditions. Among the 31 unique facial action units (FAU) identified, some showed significant differences between the two air pollution and noise exposure conditions, especially during cognitive tests, while changes in physiological parameters were minor. We also established a model for predicting dissatisfaction with overall IEQ based on facial and physiological data and achieved an accuracy of 0.71, despite the weak correlation between individual FAUs and self-assessed perceptions. Our findings underscore the potential and the challenges of innovative methods for characterizing human responses to indoor environmental stimuli. • Automatic facial expression recognition utilized under two environmental conditions. • Contrasting conditions: air change 0.2 h −1 , paint, and noise vs. 10 h −1 , HEPA filter, no paint or noise. • Significant changes in a few facial expressions observed with air pollution and noise exposure. • Minor effects observed on physiological parameters monitored using wearable sensors. • IEQ dissatisfaction predicted based on facial and physiological data with reasonable accuracy (0.71).