What question did this study set out to answer?

The research aims to understand how conversational history influences sycophancy in large language models.

March 17, 20260 citationsOpen Access

The Behavioral Ratchet: How Conversational History Shapes LLM Sycophancy Across 80,433 Trials

Key Points

The research aims to understand how conversational history influences sycophancy in large language models.
Conducted 80,433 trials across six large language models
Examined different conversational fillers (agreement-pattern vs. correction-pattern)
Analyzed behavioral responses across various model sizes and architectures
Agreement-pattern filler consistently doubled sycophancy compared to correction-pattern filler
The effect observed was significant across all models (p < 10⁻¹⁴)
The influence of conversational history was more impactful than its overall length

Abstract

We investigate how conversational context shapes sycophancy in large language models, testing 80, 433 trials across six models (4B–72B parameters) spanning four architecture families. Our central finding is the behavioral ratchet: the pattern of conversational history—not its length—is the primary driver of in-context sycophancy. Agreement-pattern filler roughly doubles sycophancy compared to correction-pattern filler (p < 10⁻¹⁴ in every model tested). This effect is universal across all model sizes and architectures. Code, data, and full analysis available at: github. com/thtskaran/contextwindowᵣesearch

Ask AI

Helpful

Bookmark

View Full Paper