Explaining Anxiety Prediction in Psychotherapy Transcripts: The Role of Patient Linguistic Features and Theoretical Constructs

Linguistics can be a helpful tool when researching psychological processes and symptoms. This study aimed to predict anxiety severity from patient language in psychotherapy transcripts. In contrast to prior work focusing on isolated feature types, we combine theory-driven psychological constructs with state-of-the-art NLP and machine learning techniques to enhance both performance and interpretability. Specifically, we asked (1) how well anxiety can be predicted, (2) which models perform best, (3) which features are most important. We extracted LIWC features, unigrams and bigrams, Transformer emotions, and topics from 529 psychotherapy transcripts from 118 patients. In addition, we constructed theory-driven features to measure negative self-focused attention, self-insight, future focus of perceived threat, and uncertainty avoidance. Each feature set was modeled using multiple machine learning algorithms. To gain insights into the most informative predictors of anxiety, eXplainable Artificial Intelligence was applied. The Unigram-Bigram Model achieved the best predictive performance (r = .77; 95 %-CI = .75-0.80). However, the Anxiety Process Model achieved notable predictive accuracy despite having only four interpretable, theory-based features. Features related to leisure, social relationships, and insecurity were associated with lower anxiety severity, health-related features and "certainty words" (e.g., totally) with higher severity. Our findings highlight a trade-off between performance and interpretability. Unigram-bigram models maximized predictive accuracy, whereas theory-driven constructs provided clinically meaningful insights into core psychological processes. Identifying predictive linguistic features, especially those linked to psychological theory, may guide future research on feedback systems and clinical applications by providing interpretable and theory-aligned insights.