Adaptive Bayesian-ML Covariance Shrinkage: A Machine-Learning-Guided Framework for Dynamic Estimation Error Control in Mean-Variance Portfolio Optimization

Mean-Variance Optimization (MVO) suffers critically from estimation error, particularly in the covariance matrix. Existing remedies such as Ledoit-Wolf shrinkage and Bayes-Stein estimation apply fixed shrinkage intensities that are blind to prevailing market conditions. This paper introduces the Adaptive Bayesian-ML Covariance Shrinkage (ABMCS) framework, which trains a gradient-boosted ensemble to predict the optimal shrinkage intensity and target dynamically, conditional on detected market regime. Empirical validation on a 10- year, 20-asset synthetic multi-regime dataset demonstrates ABMCS achieves lower portfolio turnover (−4.0% vs Ledoit-Wolf), reduced maximum drawdown (−1.8%), and superior Sharpe ratio in Bull regimes (+0.135 vs Ledoit-Wolf) with 81% regime detection accuracy. Key contributions: (1) regime-conditioned shrinkage target selection; (2) ML-predicted optimal shrinkage intensity; (3) transaction-cost-aware training objective; (4) full feature-importance explainability for governance compliance.