Language-Based Mental Health Assessments Do Not Perform Evenly Across Racial and Ethnic Groups

Psycholinguistic tools that estimate psychological symptoms from language alone aim to provide unobtrusive assessments without needing questionnaires or clinicians. Researchers largely assume that these assessments function equivalently across demographic groups. We tested this assumption in a racially diverse sample of U.S. adults (N = 1,685; Black, Asian, Latino/Hispanic, White) who described their mental health and completed standard symptom questionnaires. Language-based markers of internalizing symptoms were often group-specific: First-person pronoun use predicted symptoms only among White participants, and sentiment varied in how strongly it tracked symptoms across groups. Machine learning models trained on White participants' language produced higher prediction error for other groups than those groups' own models did. Interpretability techniques revealed that White-trained models emphasized self-focused, agentic language more than minority groups' models. These findings indicate that linguistic tools cannot assume generalizability and must be evaluated for the populations they serve.