Between-subjects study demonstrates improved perceived usability in users performing mixed-reality tasks, suggesting affect-aware dialogue enhances interaction quality.
Key Points
To evaluate the impact of prosody-based affect-aware conversational adaptation on interaction quality, usability, and task performance during mixed-reality procedural guidance.
Engineered a modular client–server mixed-reality architecture combining large language model dialogue with parallel semantic and vocal prosody affective pipelines.
Conducted a between-subjects evaluation with 40 participants performing a guided mixed-reality laboratory chemical procedure using either an affect-adaptive or non-adaptive agent.
Assessed interaction outcomes using standardized usability scales, interaction log metrics (completion time and conversational turns), workload measures, and qualitative feedback.
Affect-adaptive conversational interaction yielded significantly higher perceived usability scores compared with the non-adaptive control condition.
Participants using the adaptive agent engaged in a higher number of conversational turns and had longer task completion times, while overall workload showed no significant difference.
Qualitative assessments indicated that prosodic emotion-aware adaptation made the conversational guidance feel more natural and supportive during procedural execution.