What question did this study set out to answer?

The aim is to extend Dense Associative Memory to unit sphere structures to enhance memory capacity and retrieval dynamics.

March 28, 2026Open Access

Theory of Directional Associative Memories: Dense Hopfield Networks on the Unit Sphere S²

Key Points

The aim is to extend Dense Associative Memory to unit sphere structures to enhance memory capacity and retrieval dynamics.
Introduced a model called Directional Associative Memory on the unit sphere.
Stored patterns as arbitrary unit vectors within the sphere.
Implemented retrieval dynamics through gradient flow projected on the tangent plane of the sphere.
Renormalized vectors after each Euler step.
Achieved exponential memory capacity verified empirically up to 256 units.
Demonstrated 100% recall for non-polar patterns at capacity level alpha of 1.56.
Showed noise robustness, maintaining 100% recall with up to 36% bit flips.
Developed a novel spherical attention mechanism linked to Transformer models.

Abstract

We extend Dense Associative Memory (DAM) from binary state spaces to the unit sphere S² ⊂ ℝ³, introducing Directional Associative Memory (DAM-S²). Each neuron carries a unit vector vᵢ ∈ S², patterns are stored as arbitrary unit vectors, and retrieval dynamics follow gradient flow projected onto the tangent plane of S² with renormalization after each Euler step. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ CONTEXT: ASSOCIATIVE MEMORIES ON SPHERES This is the companion paper to "Dense Associative Memory on S¹ with injection-locking dynamics" (DOI: 10.5281/zenodo.18800042). Together they form a family of associative memories on spheres Sⁿ: ● S¹ (circle) — Phase oscillators, α* = 1.0, 7.2× classical Hopfield capacity. Basis for the S1 memory layer in PGAN. ● S² (sphere, this work) — Directional patterns, α = 1.56 (all tested), √3 SNR advantage over binary. Basis for the S2 attention layer in PGAN, connecting Dense AM directly to Transformer attention via spherical softmax. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ KEY RESULTS ● Exponential capacity for F = eᵐ, verified empirically up to N = 256 ● Non-polar patterns: 100% recall at α = 1.56 (P=100, N=64) ● Noise robustness: 100% recall up to ~36% bit flips ● Spherical attention = one-step discrete update = softmax-weighted spherical interpolation ● Logic gates on S² with two-anchor framework; ROT(θ) gate unique to S²; connection to quaternion arithmetic ● Turing completeness inherited from S¹ for binary computation ● Generalization to Sⁿ; connection to von Mises-Fisher distributions ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ CONTENTS Paper (PDF + LaTeX source), Python implementation (DenseS2AM class), 14 unit tests, 7 benchmarks with reproducible results, and all figures.

Theory of Directional Associative Memories: Dense Hopfield Networks on the Unit Sphere S²

Key Points

Abstract

Cite This Study