December 5, 2025Open Access

Is ChatGPT conservative or liberal? A novel approach to assess ideological stances and biases in generative LLMs

Key Points

GPT output is influenced by linguistic variation, reflecting conservative bias in Polish and liberal bias in Swedish.
Comparative analysis highlights how ideological bias arises from both training data and algorithms.
Findings suggest improvements in bias reduction require high-quality, curated training data for AI models.
Ongoing research is essential to comprehend the ideological stances of generative AI, especially in politically sensitive contexts.

Abstract

Abstract Extant work shows that generative AI such as GPT-3.5 and perpetuate social stereotypes and biases. A less explored source of bias is ideology: do GPT models take ideological stances on politically sensitive topics? We develop a novel approach to identify ideological bias and show that it can originate in both the training data and the filtering algorithm. Using linguistic variation across countries with contrasting political attitudes, we evaluate average GPT responses in those languages. GPT output is more conservative in languages conservative societies (polish) and more liberal in languages used in liberal ones (Swedish). These differences persist from GPT-3.5 to GPT-4. We conclude that high-quality, curated training data are essential for reducing bias.

Ask AI

Helpful

Bookmark

View Full Paper