Emotion Recognition Training Across Sensory Modalities: Comparing the Effects of Multisensory and Unisensory Training

Emotion-recognition training is widely used as a tool to improve emotion recognition and by extension social competence, yet few studies compare unisensory and multisensory trainings. We randomly assigned adults to auditory, visual, audiovisual, or control training groups using a validated protocol adapted across modalities, and assessed emotion recognition using auditory, visual, and audiovisual tests. Responses were scored with a similarity-based metric, and change was indexed with a normalized gain difference score. Primary analyses tested training-group effects on overall and modality-specific difference scores and evaluated cross-modal generalization (improvement on untrained modalities) and trained-modality advantage (improvement on the trained modality relative to the other two). All trained groups showed reliable pre- to post-test improvement across modalities. Between-group comparisons indicated that the auditory group outperformed control on the overall measure, and the audiovisual group outperformed control on the audiovisual task. Cross-modal generalization was evident in every trained group and absent in control, indicating transfer beyond the trained modality. Trained-modality advantage was not detected, suggesting more generalized training benefits. An exploratory comparison of basic and social emotions revealed a main effect of emotion type, without differences by training condition. Taken together, the results indicate that brief training strengthens perceptual cue use and cue integration, yielding gains within and across modalities. Practically, training programs that include audiovisual components may maximize benefits while also supporting generalization beyond the trained setting.