Evaluation of Ranking & Matchmaking in Competition

AbstractIn recent years, there has been an increasing amount of complaints in the video game commu- nity regarding ranking and competition. Ranking systems can be evaluated with empirical study on game dataset. For sport and competition, there is a simple model suitable for simula- tion based research. Yet this approach is relatively absent from literature. The purpose of the thesis is double. First, I introduce the matchmaking overfitting hypothesis. I perform com- parative studies between state-of-the-art rating systems, highlighting differences in behavior in challenging settings. In a second step, I present the experiment to a panel of experts in competition to measure how receptive they are to simulation.