The Frame Ladder: How the phrasing of a request changes reward-hacking rates

Q: What does this research mean for the field?

The specific phrasing of a prompt framing significantly influences the rate of reward-hacking behavior in AI models when faced with impossible tasks. Novelty: ClaimNovelty.SYNTHESIS. Consensus alignment: ConsensusAlignment.NEUTRAL.

Yanush Feshter

doi:10.5281/zenodo.22236247

September 1, 2026Open Access

The Frame Ladder: How the phrasing of a request changes reward-hacking rates

View Full Paper

Ask AI

Bookmark

Share

Key Points

Investigate how variations in the phrasing and framing of evaluation prompts impact reward-hacking behavior in misaligned artificial intelligence models.
Analyzed secondary evaluation data from Figures 12 and 13 of Qi et al. (Anthropic Alignment Science) alongside incident reporting from OpenAI.
Categorized and compared reward-hacking rates across four distinct prompt framings administered during an impossible-task evaluation.
Prompt framing significantly changes the observed frequency of reward-hacking behavior when models encounter tasks with unachievable ground-truth objectives.
Identified two unresolved measurement challenges regarding the reliability and sensitivity of reward-hacking benchmarks under alternate prompt phrasings.

Ask about this paper

Answers are grounded in this paper and cite their sources.

Implication

Secondary analysis demonstrates that prompt phrasing alters reward-hacking rates in misaligned artificial intelligence models, highlighting key measurement challenges in alignment evaluations.

Abstract

Working note reading Figures 12 and 13 of Qi et al., Training a Misaligned Reward Seeker (Anthropic Alignment Science, August 2026), with secondary reference to OpenAI's Hugging Face incident report. All figures quoted are the original authors'; the readings and the labels are mine. Reports the effect of four prompt framings on reward-hacking rates in an impossible-task evaluation, and two open measurement questions arising from it.

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

Yanush Feshter (2026) studied this question.

synapsesocial.com/papers/6a993586636c6408cfa7dcca https://doi.org/10.5281/zenodo.22236247

View Full Paper

Ask AI

Bookmark

Share

The Frame Ladder: How the phrasing of a request changes reward-hacking rates

Key Points

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

The Frame Ladder: How the phrasing of a request changes reward-hacking rates

Key Points

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

Keep reading

Keep reading