HON: A Harmonic Oscillator Network for Temporal Memory in Sequence Modeling

We present HON (Harmonic Oscillator Network), a linear hybrid sequence modeling architectureinspired by the physics of damped harmonic oscillators. The architecture integrates three parallel functionalbranches within each block:1. LACT (Local Adaptive Causal Transform) A content-dependent multi-kernel causal convolutionfor capturing local n-gram patterns.2. HON (Harmonic Oscillator State Space) A continuous-time temporal memory layer parameteriz‐ing damped harmonic oscillations evaluated via FFT-based causal convolutions.3. ASSOC (Linear Associative Memory) A normalized linear attention mechanism operating via fastcumulative prefix sums.These representations are combined using a dynamic learnable gating mechanism. Initial evaluations onWikiText-2 demonstrate an 18.5% improvement in validation perplexity compared to an equivalentTransformer baseline under identical parameter budgets. Additional experiments on Tiny ImageNet showthat the architecture successfully learns 2D spatial representations formatted as 1D sequences.We do not claim that HON replaces Self-Attention or outperforms state-of-the-art models on large-scalebenchmarks. The primary goal of this technical report is to document an initial experiment and share thefindings with the open-source community. We also acknowledge significant limitations in our current im‐plementation, including generation quality, computational overhead, and the need for more careful fusionmechanisms.