A Geometric Approach to Perceptron Learning

This work presents a coordinate-invariant geometric formulation of the perceptron, the foundational unit of neural computation. We model the perceptron as a linear map between tangent spaces of distinct parameter and observation manifolds, and the non-linear activations are interpreted as pullbacks that induce position-dependent local metrics on the parameter space. In doing so we make explicit some of the geometrical structures that are implicit in standard Euclidean implementations and the ad hoc procedures commonly employed in practice. In this formulation, the trained state emerges as an equilibrium characterised by covariant normal equations, and the efficient training as an equation of dissipative geodesic flow on the parameter manifold.This formulation recovers several established results in the optimization domain, notably Amari’s natural gradient as an overdamped limit of this motion. Conversely, in the under-damped regime, the equation gives rise to heavy-ball momentum - revealing the established method used to navigating the curvature bottlenecks. Together, these results unify normal equations, natural gradient, and momentum within a single geometric description, providing a rigorous basis for extensions to stochastic, multi-layer, and physically motivated learning dynamics.