What question did this study set out to answer?

This research aims to elucidate how in-context learning and fine-tuning function as working memory and long-term memory in language models.

May 16, 2026Open Access

In-Context Learning as Working Memory, Fine-Tuning as Long-Term Memory: A Substrate-Mechanistic Account of LLM Practice

View Full Paper

Ask AI

Bookmark

This research aims to elucidate how in-context learning and fine-tuning function as working memory and long-term memory in language models.
Conducted experiments on Qwen2.5 language models with varying scales and fine-tuning budgets.
Analyzed performance across cloze retrieval tasks and alternative route signal compression due to gradient pressure.
Investigated empirical confirmations across multiple studies to validate substrate-mechanism differences.
ICL outperformed LoRA fine-tuning on cloze retrieval by 16–28 percentage points.
FT-trained models showed decreased application accuracy below the no-context baseline.
The per-token competing-routes signal significantly collapsed with increasing gradient passes, evidencing the compression effect.

Discussion

No takes yet. Share an insight, caveat, or question.

Implication

Randomized trial demonstrates the differences in accuracy between in-context learning and fine-tuning in language models, indicating distinct operational modes.

Cite This Study

Tomas Pødenphant Lund (2026) studied this question.

synapsesocial.com/papers/6a080a29a487c87a6a40c072 https://doi.org/10.5281/zenodo.20187352

View Full Paper

Ask AI

Bookmark

Implication

Randomized trial demonstrates the differences in accuracy between in-context learning and fine-tuning in language models, indicating distinct operational modes.

In-Context Learning as Working Memory, Fine-Tuning as Long-Term Memory: A Substrate-Mechanistic Account of LLM Practice

Key Points

Discussion

Implication

Cite This Study

In-Context Learning as Working Memory, Fine-Tuning as Long-Term Memory: A Substrate-Mechanistic Account of LLM Practice

Key Points

Discussion

Implication

Cite This Study

Also Consider

Also Consider