Sub-Quadratic Recurrent Kernel Attention: Bounding Computational Complexity in Deep Contextual Sequence Representations

The quadratic arithmetic cost of conventional dense softmax attention motivates interest in alternativemechanisms for long-context sequence modeling. This conceptual manuscript outlines Sub-QuadraticRecurrent Kernel Attention (SRKA), a proposed causal attention design combining a non-negativefeature representation of queries and keys with a recurrent key-value summary. The formulation avoidsexplicitly constructing an N-by-N attention matrix and maintains a recurrent state whose dimensionsdepend on feature width and value dimension rather than the number of processed tokens. We specifythe proposed state updates, identify the assumptions under which sequence-length-linear arithmeticcost is plausible, and examine key open issues: approximation quality relative to softmax, numericalstability, gating semantics, and efficient parallel training. The proposal is situated within existing workon Transformers, linear attention, Performer, FlashAttention, and state-space sequence models. Noimplementation, formal error proof, dataset evaluation, or measured performance comparison isreported. The contribution is a clearly delimited research design and validation agenda rather than anempirically established advance