Multi-Fidelity Reinforcement Learning with Control Variates

In this paper, we investigate reinforcement learning (RL) in multi-fidelity environments and enhance the performance of the agent using cross-correlated data.We introduce a multifidelity estimator based on control variates to reduce the variance in state-action value function estimation.By employing this estimator, we develop a multifidelity Monte Carlo RL (MFMCRL) algorithm that boosts agent learning in high-fidelity settings.Our experiments show that, given a finite highfidelity sample budget, the MFMCRL agent outperforms an RL agent relying solely on high-fidelity interactions for policy optimization.