HARL-TRADE: A hierarchical adaptive reinforcement learning framework for second-level high-frequency trading

High-frequency trading (HFT) demands adaptive strategies to navigate volatile markets. Current cutting-edge discrete sub-agent frameworks struggle with rigid market condition allocations, limiting adaptability. We propose a hierarchical framework with an attention-based meta-agent for dynamic sub-agent coordination. By leveraging market embeddings and reinforcement learning, the meta-agent optimally adjusts responsibility weights, enabling adaptive action aggregation across market regimes. Experiments on historical second-level HFT data show that the proposed framework outperforms state-of-the-art baselines, achieving a 42.15% total return and a 4.19 Sharpe ratio. Ablation studies validate the contributions of the dynamic sub-agent assign mechanism and multi-head attention mechanism, highlighting the framework’s ability to adapt to market transitions and deliver superior performance.