EXPRESS: Salesperson Attractiveness Beyond Looks in Livestreaming E-commerce: Mixed Method of Multimodal Machine Learning and Explainable AI

Salesperson attractiveness (SA) plays a pivotal role in marketing; however, existing research has focused mainly on SA derived from physical cues, overlooking its non-physical aspect. To fill this gap, two studies were conducted in the livestreaming context. Study 1 focused on SA prediction and interpretation. Using a dataset of 1,933 livestreaming videos, this research customized a multimodal machine learning (MMML) model with a marketing metric (product category). The customized MMML model integrates the salesperson’s verbal, vocal, and visual cues, as well as their cross-modal interactions, achieving the highest prediction accuracy of 92.17% in predicting SA. An explainable AI model was subsequently applied to identify how specific verbal, vocal, and visual cues contributed to the prediction of SA. Study 2 explored the sales effect of SA and its boundary condition. The results indicate a U-shaped relationship between SA and sales, with this relationship being more pronounced for experience products than for search products. Overall, this research offers a more comprehensive understanding of SA by integrating both physical and non-physical cues, establishing an empirical framework to predict and interpret SA and providing marketers with novel insights into its effect on livestreaming e-commerce sales.