Risk-Sensitive Portfolio Optimization under Regime Shifts: A Hierarchical Deep Reinforcement Learning Approach

Financial market time series are characterized by severe non-stationarity, shiftingvolatility regimes, and low signal-to-noise ratios. Standard continuous Deep Reinforcement Learning (DRL) algorithms (e.g., PPO, DDPG, A2C) frequently fail in real-worlddeployment on leveraged financial markets. We identify a fundamental domain vulnerability: the Friction-Regime-Leverage Paradox. Exposed to noisy price series underrealistic brokerage spreads, continuous neural policies commit a severe transactioncost-induced performance collapse (informally termed “Transaction-Fee Bankruptcy”)by hyperactively whipsawing to capture microstructural noise. Furthermore, underinstitutional leverage (5.0x), standard policies suffer catastrophic drawdown and totalmargin liquidation when structural volatility regimes shift. To address this systemicvulnerability, we propose a hierarchical, regime-adaptive deep reinforcement learning model, utilizing a hierarchical partition of control to regularize the neural policy’s action space. Specifically, our approach separates execution control: (1) a low-level discrete statistical filter acts as a strict execution noise-gate, filtering microstructural noiseand neutralizing transaction fee churn; and (2) a high-level volatility regime routingpool acts as a neural meta-policy to dynamically optimize continuous risk parameters(active leverage limits, capital allocation weights, and dynamic stops) in real-time. Weevaluate this methodology empirically on daily and hourly Gold (GLD) and Crude Oil(USO) data from 2017 through May 2026. While standalone continuous reinforcementlearning policies undergo transaction-cost-induced performance collapse (-30.11% loss),the proposed hierarchical model delivers a highly consistent out-of-sample walk-forwardreturn of +3,466.19% (Sharpe 1.043) under strict leverage constraints, suggestingsignificant risk-adjusted value-add over standard benchmarks.