Reinforcement Learning in Multi-Asset Trading: Integrating Temporal-Frequency Features and Novel Risk-Adaptability Indices

Financial markets present a significant challenge for traditional trading models, which often fail to capture the complex, non-stationary nature of asset price movements. This study introduces a novel reinforcement learning (RL) framework designed to overcome these limitations by integrating signal processing with advanced artificial intelligence (AI). We enhance the state representation for RL agents by extracting temporal-frequency features from market data using the Short-Time Fourier Transform (STFT). We then conduct a rigorous comparative analysis of three leading RL algorithms—Deep Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO)—across the diverse markets of gold, oil, and Bitcoin, covering the period 2014–2024. To provide a more sophisticated evaluation of performance, we propose two novel metrics: the Robust Trading Strategy Score (RTSS), which balances profitability with downside risk, and the Market Adaptability Index (MAI), which measures an agent’s ability to adapt its strategy to changing market dynamics. Our empirical results, validated by comprehensive statistical testing (MANOVA, Tukey’s HSD Test, and Kruskal-Wallis Test), demonstrate that the PPO agent consistently and significantly outperforms its counterparts in both profitability and risk-adjusted returns. This work provides a robust framework for developing next-generation, AI-driven trading systems and offers new tools for the rigorous evaluation of their performance.