Codivergences and information matrices

We propose a new concept of codivergence, which quantifies the similarity between two probability measures P₁, P₂ P 1 , P 2 relative to a reference probability measure P₀ P 0 . In the neighborhood of the reference measure P₀ P 0 , a codivergence behaves like an inner product between the measures P₁-P₀ P 1 - P 0 and P₂-P₀ P 2 - P 0 . Codivergences of covariance-type and correlation-type are introduced and studied with a focus on two specific correlation-type codivergences, the χ ² χ 2 -codivergence and the Hellinger codivergence. We derive explicit expressions for several common parametric families of probability distributions. For a codivergence, we introduce moreover the divergence matrix as an analogue of the Gram matrix. It is shown that the χ ² χ 2 -divergence matrix satisfies a data-processing inequality.