Not one size fits all: model-specific effects of reasoning on emotion recognition from body-gesture videos

Background The growing use of generative artificial intelligence (GAI) in mental-health contexts underscores the need to evaluate its emotion-recognition capacity. Objective This study examined whether a reasoning-oriented model or structured Chain-of-Thought (CoT) prompting improved recognition of emotions conveyed through bodily gestures. Methods Four Gemini models were evaluated using 36 silent videos depicting 18 emotions. Each model completed 20 trials per video under Zero-Shot and CoT prompting, yielding 5,760 responses. Population-averaged generalized estimating equation models compared accuracy across models, prompts, emotional groups, and the human benchmark. Results Gemini 2.0 Pro showed higher accuracy than the other models following multiplicity adjustment, whereas Gemini 2.0 Flash Thinking did not outperform the general-purpose models. The marginal CoT-versus-Zero-Shot contrast was not statistically significant. However, the Model × Prompt Type interaction was significant. Accuracy was higher under the tested CoT condition than under Zero-Shot prompting for Gemini 2.0 Pro and lower under the tested CoT condition for Gemini 2.0 Flash. The Model × Prompt Type × Emotional Group interaction was not significant. The Performer × Emotional Group interaction was significant, although within-model differences between positive and negative emotions did not survive adjustment. Three models showed lower accuracy than humans for negative emotions, whereas no significant model–human differences were found for positive emotions. Conclusion The data did not support a general improvement in emotion recognition under CoT prompting. Prompt-condition differences varied across models. This pattern underscores the need to evaluate reasoning prompts for specific models and tasks before application in mental-health contexts.