Answers are grounded in this paper and cite their sources.
Comprehensive analysis reveals significant preference bias in LLM dialogue planners, indicating challenges for ethical deployment.
Large language models have facilitated the development of sophisticated dialogue planner policies. However, their reliance on LLM-based simulation and feedback for policy optimization may introduce preference bias. This paper presents the first comprehensive analysis of preference bias in LLM-based dialogue planners. It systematically evaluates four state-of-the-art planning methods across three dialogue domains, utilizing multiple LLM families at varying scales. Our investigation reveals that all tested planners exhibit significant preference bias, favoring narrow strategy sets over maintaining balanced distributions. Notably, larger models paradoxically amplify bias. User simulation emerges as the primary bias driver, while diverse persona simulation proves ineffective as a mitigation strategy. Most concerning, preference bias leads planners toward ethically problematic strategies that achieve short-term success while compromising real-world effectiveness and ethical standards. These findings establish fundamental challenges for responsible deployment of LLM-based dialogue systems and provide crucial insights for developing more reliable and ethically aligned planning approaches.
No takes yet. Share an insight, caveat, or question.
Yang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: