Exploring addressee styles in spontaneous conversations

During conversations, both the main speaker and their addressee work together to co-construct the interaction. However, addressee reactions, referred to as feedback, are often variable, making them challenging to model and predict. To better understand this variability, we introduce the concept of addressee style, defined as the distinctive multimodal behaviors exhibited by addressees during conversations. Our study focuses on (i) variability in addressee styles across conversational activities and over time, (ii) convergence of styles within dyads, and (iii) correlations with storytelling quality and engagement. Using the SMYLE corpus, which features French face-to-face storytelling and free conversation, we examined generic feedback (e.g., ”mh”, ”yeah”, nods) and specific feedback (evaluative responses like laughter, speech e.g., ” it's amazing ,” facial expressions, etc.). Feedback components that include type of verbal responses, head movements, facial expressions, hand gestures, pitch, and loudness were used to examine each participant's average feedback behavior. Hierarchical clustering revealed three addressee styles: Low-Profile (mainly generic and gestural feedback), Balanced (rich, multimodal, and lengthy feedback, with a balance of generic and specific responses), and Intense (mainly multimodal specific feedback). Analysis of intra-individual variability showed that addressee styles vary across conversational activities but remain relatively stable within free conversation. Additionally, results suggest that participants tend to adapt their addressee styles to align with their conversational partners. We found no significant correlation between addressee style and storytelling quality, suggesting that effective interaction can emerge from diverse listening approaches rather than a single ”ideal” style. However, results suggested a potential link between addressee style and engagement.