Answers are grounded in this paper and cite their sources.
Survey identifies how fractional calculus can improve language models' performance, indicating future research directions.
Large language models (LLMs) face critical bottlenecks like the “memory wall” and context degradation due to standard integer-order Markovian assumptions. This survey reviews integrating fractional calculus to explore potential solutions. Synthesizing the recent literature reveals a transitional landscape: while fractional mechanisms yield significant empirical gains in medium-scale architectures (e.g., up to 14.36% absolute accuracy improvements), explicit validation on autoregressive models exceeding one billion parameters remains largely unexplored. By introducing heavy-tailed power-law memory and non-local historical gradients, fractional operators help mitigate catastrophic forgetting and optimization instability. Furthermore, we propose dual-integration frameworks: a decoupled neuro-symbolic architecture for scientific computing, and an endogenous fractional evolution architecture for native long-context reasoning. While fractional dynamics offer mathematical pathways to alleviate LLM constraints, computational overhead and training stability challenges remain. Future research should prioritize selective fractionalization and data-driven scientific discovery.
No takes yet. Share an insight, caveat, or question.
Tu et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: