Geometry of Statistical Manifolds

A statistical manifold M can be defined as a Riemannian manifold each of whose points is a probability distribution on the same support. In fact, statistical manifolds possess a richer geometric structure beyond the Fisher information metric defined on the tangent bundle TM. Recognizing that points in M are distributions and not just generic points in a manifold, TM can be extended to a Hilbert bundle HM. This extension proves fundamental when we generalize the classical notion of a point estimate—a single point in M—to a function on M that characterizes the relationship between observed data and each distribution in M. The log likelihood and score functions are important examples of generalized estimators. In terms of a parameterization θ: M→Θ⊂Rk, θ^ is a distribution on Θ while its generalization gθ^=θ^−Eθ^ as an estimate is a function over Θ that indicates inconsistency between the model and data. As an estimator, gθ^ is a distribution of functions. Geometric properties of these functions describe statistical properties of gθ^. In particular, the expected slopes of gθ^ are used to define Λ (gθ^), the Λ-information of gθ^. The Fisher information I is an upper bound for the Λ-information: for all g, Λ (g) ≤I. We demonstrate the utility of this geometric perspective using the two-sample problem.