Integrating multi-modal emotion recognition with strategy generation: a transformer approach to sustainable marketing

This paper proposes the multi-modal emotion perception and strategy optimisation transformer (MEPSO-Transformer), a dual-task architecture designed to jointly perform emotion classification and green marketing strategy recommendation based on heterogeneous user data.The model incorporates modality-specific encoders, hierarchical gated cross-attention fusion, and two parallel decoders, and is trained under a multi-task learning scheme.To evaluate its effectiveness, experiments are conducted on three datasets: CMU-MOSEI, MELD, and a constructed domain-specific dataset, GreenPromo-Emotion, which contains 6,200 multi-modal samples annotated with seven emotional categories and five predefined marketing strategies.Results show that MEPSO-Transformer achieves an average accuracy of 84.3% and a macro-F1 score of 81.2% on CMU-MOSEI, outperforming the best baseline by +2.3% and +2.6% respectively.On MELD, the model attains a 75.0%macro-F1 and a Hit@1 score of 65.7%.These findings demonstrate the model's applicability to sustainable marketing tasks requiring real-time affective understanding and strategy selection.