Deep Reinforcement Learning with Permutation Equivariant Neural Network for Multi-Armed Bandit

Permutation equivariance (PE) is a property widely present in mathematics and machine learning. Classic deep reinforcement learning (DRL) algorithms, such as Deep Q-Network (DQN), require thorough exploration of the state space to achieve optimal performance. For a PE problem such as the Multi-Armed Bandit (MAB), the PE property helps reduce the space that needs to be explored. This paper proposes PEDQN, a PE DRL framework based on DQN that applies a PE neural network structure. Our MAB experiments show that PEDQN has clear advantages over DQN with a fully connected network and achieves the same or better performance than UCB1 when tested in the same environment as the training. Additionally, PE-based architectures can adapt to varying input sizes, with pretrained models accelerating learning in new environments.