Multimodal computational analysis of longitudinal stress profiles in healthcare workers

Healthcare workers (HCWs) face elevated risk for burnout and stress-related disorders, yet conventional self-report assessments are limited by professional stigma and underreporting. We evaluated whether multimodal computational analysis of brief, remotely collected stress narratives could distinguish longitudinal stress profiles. In a prospective six-week cohort study, N = 750 HCWs were invited to provide weekly naturalistic narratives describing recent stressful experiences and repeated measures of anxiety, depression, burnout, and subjective distress. Among 553 participants with sufficient repeated self-report data, multivariate longitudinal clustering identified two distinct profiles: Resilient ( n = 295) and Vulnerable ( n = 258). Linguistic, acoustic, and facial expression embeddings were extracted from the recordings and integrated using a hierarchical multimodal transformer. In a held-out test set, the model achieved an AUROC of 0.75, outperforming unimodal (linguistic embeddings; AUROC = 0.63) and bimodal (linguistic + acoustic; AUROC = 0.70) configurations. These findings provide proof of concept that multimodal embeddings extracted from brief stress narratives are associated with occupational stress profiles, potentially complementing traditional assessment methods and informing targeted prevention strategies.