⌘+K

Pulse Explore Communities Researchers Journals

Home Communities Explore

October 1, 2025Open Access

Analyzing Dialectical Biases in LLMs for Knowledge and Reasoning Benchmarks

Eileen Pan Anna Seo Gyeong Choi Maartje ter Hoeve Skyler Seto Allison Koenecke

View Full Paper

Ask AI

Bookmark

Share

Key Points

LLM accuracy decreases by 20% when using non-standard dialect questions.
Three grammar rules significantly contribute to under-performance in various dialects.
Analyzing dialect biases in LLMs underscores the need for improved bias mitigation methods.
Individual grammatical rules affect LLM performance variably, demanding targeted analysis.

Implication

Analysis reveals a 20% accuracy drop for non-standard English dialects, highlighting biases in LLMs.

Abstract

No abstract yet. The journal hasn't published one we can show.

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

Pan et al. (2025) studied this question.

synapsesocial.com/papers/68e25378d6d66a53c2474329 https://doi.org/10.48550/arxiv.2510.00962

View Full Paper

Ask AI

Bookmark

Share

Keep reading

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

1Native Design Bias: Studying the Impact of English Nativeness on Language Model Performance2024 · 3 citations
2LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users2024 · 5 citations
3Easy Problems That LLMs Get Wrong2024 · 2 citations
4The Bias is in the Details: An Assessment of Cognitive Bias in LLMs2025
5Do Large Language Models Exhibit Cognitive Dissonance? Studying the Difference Between Revealed Beliefs and Stated Answers2024 · 1 citations

Language models align with human judgments on key grammatical constructions2024
Cognitive Bias in High-Stakes Decision-Making with LLMs2024
Multilingual large language models do not comprehend all natural languages to equal degrees2026