Literature review analyzes dynamical systems modeling self-attention in transformers, highlighting system convergence.
Deep neural networks can be understood as discretizing a continuous dynamical system. This literature review analyzes how the multi-particle dynamical system formulation models the self-attention mechanism in transformers. We will discover how this formulation enables the systematic study of the system's convergence towards clusters and its relation with the Kuramoto oscillator.
No takes yet. Share an insight, caveat, or question.
Yuxuan Zhang (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: