What question did this study set out to answer?

To develop a framework for measuring and quantifying emergent biases arising from conversational interactions between large language models in multi-agent environments.

August 20, 2026PLoS ONE2 citationsOpen Access

Unmasking conversational bias in AI multiagent systems

ECErica CoppolilloUniversity of Southern California GMGiuseppe MancoFoundation Center LALuca Maria AielloPioneer (United States)

Key Points

To develop a framework for measuring and quantifying emergent biases arising from conversational interactions between large language models in multi-agent environments.
Simulated small echo-chamber interactions pairing conversational large language models (LLMs) initialized with aligned viewpoints on polarizing topics.
Tracked stance shifts across generated conversational messages and evaluated the detection capabilities of standard questionnaire-based bias benchmarks.
Interacting agents exhibited significant stance shifts during dialogue, with conservative echo chambers showing marked drift toward liberal viewpoints.
Standard state-of-the-art questionnaire-based bias detection techniques failed to identify the emergent conversational biases observed in the multi-agent simulations.

Abstract

Detecting biases in the outputs produced by generative models is essential to reduce the potential risks associated with their application in critical settings. However, the majority of existing methodologies for identifying biases in generated text consider the models in isolation, overlooking the contextual dynamics of multi-agent systems. In particular, biases emerging from interactions among conversational agents remain largely unexplored. To address this gap, we present a framework designed to quantify biases within multi-agent systems of conversational Large Language Models (LLMs). Our approach involves simulating small echo chambers, where pairs of LLMs, initialized with aligned perspectives on a polarizing topic, engage in discussions. Contrary to expectations, we observe significant shifts in the stance expressed in the generated messages, particularly within echo chambers where all agents initially express conservative viewpoints, in line with the well-documented political bias of many LLMs toward liberal positions. Crucially, the bias observed in the echo-chamber experiment remains undetected by current state-of-the-art bias detection methods that rely on questionnaires. This highlights a critical need for the development of a more sophisticated toolkit for bias detection and mitigation for AI multi-agent systems.

اسأل الذكاء الاصطناعي

Bookmark

View Full Paper