Bimodal emotion decoding: When faces and voices diverge

It is well established that people infer emotions from both faces and voices, yet how they integrate these cues when the two modalities convey conflicting emotions remains unclear. Evidence for facial dominance has typically come from studies contrasting facial and vocal expressions of anger and happiness, often using binary response formats that require observers to choose between face-consistent andvoice-consistent emotion labels. These findings, however, fail to inform us whether facial dominance reflects a general principle of bimodal emotion perception or whether these results are shaped by the specific emotion pairing and response format. Across six preregistered experiments (total N = 359), participants judged dynamic face–voice expressions spanning three incongruent pairings: anger–happiness, anger–sadness, and happiness–sadness. Experiments 1–3 used multidimensional ratings, and Experiments 4–6 used open-ended descriptions. The results revealed a pairing-dependent pattern: anger–happiness pairings showed robust facial dominance effects, anger–sadness pairings showed vocal dominance effects, and happiness–sadness pairings produced more heterogeneous impressions, including blended responses and third-category interpretations (i.e., anger or disgust for sad face–happy voice pairings). Together, these findings challenge a single-modality dominance view and suggest that bimodal emotion decoding depends on the specific emotions being integrated. To explain these pairing-dependent patterns, we propose and provide initial support for an emotion-similarity account, in which the relative influence of faces and voices depends on how distinguishable the competing emotions are within each modality.