Routing in congested baggage handling systems using deep reinforcement learning

The increasing number of people choosing to travel by airplane puts pressure on the baggage handling systems in airports. As the load increases, the risk of deadlocks in the systems increase as well. Therefore, it is increasingly important to find routing solutions which can handle the high loads. Currently this is achieved by using shortest path algorithms and hand engineered site-specific routing rules, based on the experience of the employees and on trial and error processes using complex emulators. This is a time-consuming and costly approach, as every airport needs its own set of routing rules. New development within machine learning, and especially reinforcement learning allows very complex control policies to be found in large environments. This could therefore potentially solve the need of manually creating site-specific routing rules. This paper proposes to use a single global deep reinforcement learning agent to route a fleet of baggage-totes to continuously pick up and deliver baggage in simple yet functionally realistic simulations of baggage handling systems. This is achieved using a Dueling DQN architecture with prioritized experience reply and a multi action approach. Training and testing are performed in three baggage handling system environments of different size and complexity. The results show that by training with a broad distribution of loads, it is possible to get a model, capable of routing in highly congested baggage handling systems. The results also show that the reinforcement learning agent can limit the number of deadlocks up until a higher load than both a static shortest path and a dynamic shortest path method, even if the dynamic shortest path method is using a naive deadlock avoidance add-on.