On batch normalization for SPDnet

This paper deals with batch normalization for SPDnet, a deep learning architecture specifically designed to handle covariance matrices. Current SPDnet batch normalization relies on geometric means, which require expensive iterative computations and lacks formal backpropagation derivations. These limitations are addressed through two contributions: (i) formal backpropagation derivation for SPDnet batch normalization, and (ii) computationally efficient closed-form alternatives to the geometric mean: log-Euclidean, arithmetic and harmonic means, and the geometric mean of arithmetic and harmonic means. Simulation results on the batch normalization layer with different means demonstrate significant reductions in memory consumption and comparable training time by avoiding automatic differentiation, demonstrating the effectiveness of our approach. Numerical experiments on three real datasets show that simpler means can outperform the geometric mean with up to 5% accuracy improvements while reducing training time by a factor of 5.