Q-Learning-based Operator Selection for Energy-Efficient Non-permutation Flow Shop Under Time-of-Use Electricity Tariffs

In light of growing environmental concerns and rising energy costs, the manufacturing industry is increasingly focusing on energy-efficient scheduling. This paper addresses the Non-Permutation Flow Shop Scheduling (NPFS) problem under Time-of-Use (TOU) pricing, where job sequences may vary across machines. The objective is to minimize the Total Energy Cost (TEC) while maintaining a constraint on the makespan to remain within the time horizon, thereby balancing production efficiency and sustainability. To achieve this, VNS-QL, a hybrid algorithm, is proposed, which uses Q-learning for operator selection within the Variable Neighborhood Search framework. The algorithm is evaluated through experiments on 20 challenging instances adapted from Vallada et al., 2015 and extended to incorporate TEC under TOU pricing. Results show that VNS-QL consistently outperforms VNS in reducing TEC. These findings highlight the value of integrating reinforcement learning into metaheuristic frameworks to guide the search more efficiently, achieving notable energy savings