Empirical evaluation reveals significant calibration bias of large language models across Turkish voter profiles, indicating current AI tools cannot reliably replace standard voting advice...
Large language models (LLMs) are increasingly deployed for a wide range of purposes, yet their potential to function as Voting Advice Applications (VAAs) on the basis of lifestyle profiles has yet to be examined. This study empirically tests whether LLMs can serve as reliable political guidance tools by using voter profiles constructed from nine lifestyle typologies developed by KONDA Research. These typologies classify Turkish society according to value orientations, consumption habits, socioeconomic status, and political leanings. Nine representative lifestyle profiles were submitted to five LLM platforms-ChatGPT, Microsoft Copilot, Google Gemini, Perplexity, and Claude- which together account for 99% of AI-related web traffic in Turkey, using prompts written in Turkish. The findings reveal substantial calibration divergence across models. While all models correctly identified the dominant party for politically homogeneous profiles, performance declined dramatically for heterogeneous and floating voter segments. Spearman rank correlation analysis demonstrated that, despite failing to achieve cardinal (percentage-level) accuracy, LLMs performed reasonably well in terms of ordinal (rank-order) accuracy. LLMs systematically over-represented major parties while under-representing minor ones. The study concludes that LLMs cannot reliably replace conventional VAAs in their current form, but they may still offer limited potential as a supplementary tool in hybrid VAA designs.
No takes yet. Share an insight, caveat, or question.
Gökhan Çelen (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: