Collusion by mistake: Does algorithmic sophistication drive supra-competitive profits?

A burgeoning literature shows that self-learning algorithms may, under some conditions, reach seemingly-collusive outcomes: after repeated interaction, competing algorithms earn supra-competitive profits, at the expense of efficiency and consumer welfare. This paper offers evidence that such behavior can stem from insufficient exploration during the learning process and that algorithmic sophistication might increase competition. In particular, we show that allowing for more thorough exploration does lead otherwise seemingly-collusive Q-learning algorithms to play more competitively. We first provide a theoretical illustration of this phenomenon by analyzing the competition between two stylized Q-learning algorithms in a Prisoner's Dilemma framework. Second, via simulations, we show that some more sophisticated algorithms exploit the seemingly-collusive ones. Following these results, we argue that the advancement of algorithms in sophistication and computational capabilities may, in some situations, provide a solution to the challenge of algorithmic seeming collusion, rather than exacerbate it.