What question did this study set out to answer?

This research aims to explore why fine-tuned language models exhibit higher confidence over their counterparts in in-context learning despite no accuracy benefit.

July 26, 2026Open Access

Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse

View Full Paper

Ask AI

Bookmark

This research aims to explore why fine-tuned language models exhibit higher confidence over their counterparts in in-context learning despite no accuracy benefit.
Controlled experiments conducted on Qwen2.5 models across 47 invented knowledge facts.
Evaluation of various fine-tuning and in-context learning strategies to determine confidence and calibration.
Application of statistical controls to assess knowledge installation mechanisms.
ICL outperformed LoRA-FT on cloze retrieval by 16–28 percentage points across models.
Confidence in FT-trained models lower as cumulative gradient passes increase, with entropy collapsing from 0.32 (ICL) to ≈0.00 (FT).
Installation mechanisms were validated via controlled tests showing a 97.5–100% knowledge recall when answers were not copyable from the input.

Discussion

No takes yet. Share an insight, caveat, or question.

Implication

Randomized trial investigates confidence in fine-tuned models vs. in-context learning, suggesting structural implications for language model accuracy.

Cite This Study

Tomas Pødenphant Lund (2026) studied this question.

synapsesocial.com/papers/6a65a962d3aea3239cd7935c https://doi.org/10.5281/zenodo.21520555

View Full Paper

Ask AI

Bookmark

Implication

Randomized trial investigates confidence in fine-tuned models vs. in-context learning, suggesting structural implications for language model accuracy.

Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse

Key Points

Discussion

Implication

Cite This Study

Per-Token Confidence Trajectory in LoRA Fine-Tuning: Substrate-Output Evidence for Cumulative Calibration Collapse

Key Points

Discussion

Implication

Cite This Study

Also Consider

Also Consider