Theoretical study demonstrates improved root-linear convergence in symmetric fixed-point iterations, indicating faster performance for Tyler's M-estimation.
Key Points
To establish theoretical guarantees for the root-linear convergence factor improvement achieved by windowed Anderson acceleration on symmetric linear and nonlinear fixed-point operators.
Analyzed the convergence rate of windowed Anderson acceleration (AA) using a sliding window of prior iterates for linear, symmetric, and contractive operators.
Formulated and evaluated a modified AA algorithm for nonlinear operators with symmetric, non-scalar, and locally contractive Jacobians at fixed points.
Validated theoretical findings through numerical simulations and evaluated empirical performance on Tyler's M-estimation across multiple data models.
Derived the first mathematical proof showing that windowed AA strictly improves the root-linear convergence factor over standard fixed-point iterations for linear symmetric contractive operators.
Proved an equivalent convergence factor acceleration for nonlinear operators with locally contractive symmetric Jacobians using a modified windowed AA framework.
Observed in numerical experiments that windowed AA significantly outperforms baseline fixed-point methods in convergence speed when solving Tyler's M-estimation problems.