Autonomous Multi-Drone Warehousing through Deep Reinforcement Learning and Predictive Potential Fields

Abstract The multi-agent reinforcement learning (MARL) problem is a well-known field of study that has been gaining special interest in the last decade. From cooperative to competitive learning, MARL systems are mainly validated in game-theoretical environments and real-time game engines due to the inherent system complexity. Given the robustness of Q-based methods under multi-agent frameworks proved in previous multi-agent studies, we present a real autonomous multi-drone system for warehousing that combines a double deep Q network (DDQN) for high-level control and a predictive potential field algorithm for dynamic obstacle avoidance. A linear quadratic regulator (LQR) is used for low-level vehicle dynamics. Our decentralized and generalized DDQN is trained in a virtual representation of our indoor flight arena. Both controllers are integrated and validated on a real heterogeneous fleet of 3 multirotor vehicles that need to pick up and deliver a series of assigned virtual payloads while coordinating static and dynamic collision avoidance. As compared to point-to-point (p2p) navigation, the implemented system demonstrates to be more robust and resilient against unexpected conditions. Further model training schemes and integration designs are proposed as future work to improve collective swarm behavior.