Cooperative reinforcement learning for multiple units combat in starCraft

This paper presents a cooperative reinforcement learning model to tackle the problem of multiple units combat in StarCraft. We construct an efficient state representation method to break down the complexity caused by the large state and action space in combat scenario. This method takes units' state and various distance information into consideration, including the current step and the last step. To solve the problem of sparse and delayed rewards, a reward function including small intermediate rewards is introduced. This reward function helps to balance units' move and attack, and encourages our units to fight as a team. We present gradient-descent Sarsa(À) to train the learning model, and use a neural network as the function approximator for the Q values. The experimental results presented in this paper show our controlled units can successfully learn to combat in a cooperative way, and defeat the built-in AI in a 3 Goliaths against 6 Zealots StarCraft combat scenario.