Simulation-as-a-Service for Reinforcement Learning Applications by Example of Heavy Plate Rolling Processes

In the production industry, the digital transformation enables a significant optimization potential. The concept of reinforcement learning offers a suitable approach to train agents on learning control strategies, further advancing automation. While applications training directly on real-world processes are rare due to economical and safety constraints, simulations offer a way to develop and evaluate agents prior to deployment. With the rise of service-based business models, the simulation owner and the machine learning expert are likely to be different stakeholders in a joint project. Due to different requirements for both simulations and reinforcement-learning agents, the stakeholders may be reluctant or unable to grant full access to the respective software. This poses a serious impediment to the potential of the digital transformation. In this paper, a distributed architecture is proposed, which allows the remote training of reinforcement learning agents on a simulation. It is shown that this architecture allows the cooperation between two stakeholders by exposing a suitable technical interface to the simulation. The proposed architecture is implemented for a simulation of the multi-step metal forming process of heavy plate rolling. Furthermore, the implemented architecture is used to successfully train a reinforcement-learning agent on the task of designing optimal parameter schedules.