Integrating transfer learning into multi-agent reinforcement learning for energy-efficient adaptive control of heating systems in thermoforming

This study proposes a framework that integrates Transfer Learning (TL) into Multi-Agent Reinforcement Learning (MARL) for real-time, energy-efficient thermal control of the heating stage in thermoforming processes, while concurrently minimizing energy consumption and adapting to varying manufacturing conditions. To enable a scalable framework under these conditions, a Fully Connected Neural Network-based Multi-Agent Proximal Policy Optimization (FCNN-MAPPO) architecture is developed using a multi-objective reward function for concurrently minimizing control error, thermal energy consumption, and instability, while preserving temporal dynamics through state augmentation. The resulting Multi-Agent Transfer Reinforcement Learning (MATRL) framework combines direct parameter transfer for within-scenario learning with experience sharing for cross-scenario adaptation, enabling faster convergence and improved generalization. Results show that under high convective heat transfer conditions, MATRL can reduce training time by 29.3%, decrease energy consumption by 28.1%, and improve average error by 2.02 °C compared to baseline MARL (i.e., without TL). Under elevated ambient temperature conditions, energy usage and settling time were reduced by 42.6% and 41.7%, respectively. Under synchronous multi-parameter variations (extreme conditions of convective heat transfer, sheet conductivity, and ambient temperature), MATRL maintained energy efficiency within 0.5% of nominal conditions while reducing temperature dispersion by 47%, demonstrating robust multidimensional adaptability without retraining. Statistical validation across multiple runs with random seeds showed stable performance, with a coefficient of variation of 7.3% and no divergence.