Exploring Variance in Reinforcement Learning Environment to Bridge the Sim-To-Real Gap: A Case Study in Color Mixing

This is the authors' abstract. We don't add key points for this paper.

Reinforcement Learning (RL) has proven effective in optimizing complex processes, yet its application in real-world scenarios often encounters the Sim-to-Real Gap—discrepancies in performance between simulated training and real-world deployment. This paper explores the Sim-to-Real Gap in the context of a simplified but dynamic environment: color mixing. The study focuses on training a RL agent to mix target colors from three ink colors, examining the effect of incrementally increasing simulation complexity on the agent’s ability to generalize to real-world scenarios. Motivated by the challenges in automating precision tasks like CAR-T cell therapy, this work investigates how the level of simulation detail impacts RL performance in real-world applications. The findings offer valuable insights into the role of simulation fidelity in achieving robust real-world performance, contributing to a deeper understanding of RL in high-precision, control-intensive tasks.