CAWN: Continuous Acoustic Wave Networks for Autoregressive Language Modeling

Modern Large Language Models (LLMs) rely heavily on the Transformer architecture and the self-attention mechanism, which scales quadratically with sequence length. Recent alternatives, such as State Space Models (SSMs), which seek to achieve linear time complexity, often suffer from signal degradation over extended contexts. In this paper, we introduce a novel, fully continuous sequence-mixing architecture: the Continuous Acoustic Wave Network (CAWN). Instead of discrete matrix-based attention, our architecture projects hidden states into multi-headed complex-domain phasors. Sequence mixing is achieved through a causal, O (L) Phase Accumulation mechanism. To prevent signal degradation over ultra-long contexts, we introduce a dual-gated Selective Phase Resonance mechanism. This incorporates Frequency-Dependent Retention, Hard-Threshold Gating via Straight-Through Estimation, and a Temporal Syntax Cache (1D-Convolution) to capture short-term local dependencies before global wave projection. Furthermore, we replace standard dense linear projections with a Depth-wise Harmonic Convolution for optimal spatial frequency mixing, augmented by Block Attention Residuals to enable depth-wise state routing. We scale this architecture to a 150-Million-parameter model, utilizing custom Triton kernels for hardware-efficient, true-complex phase accumulation in float32. Trained via a continuous streaming loop on a 100-Billion-token English corpus, the prototype is evaluated at a 5-Billion-token milestone. Subjected to a Targeted Semantic Retrieval protocol, our empirical evaluations demonstrate robust vocabulary acquisition and highly extended explicitly learned contextual denoising. By leveraging O (1) state-passing via chunked prefill, the model successfully retrieves targeted information across 2, 000, 000 tokens while strictly plateauing at 8. 72 GB of Peak VRAM, empirically overcoming the O (L²) context memory wall.