Procedural Generation of Game Maps With Human-in-the-Loop Algorithms

A key challenge in procedural content generation is to automatically evaluate whether the generated content has good quality. In this paper, we describe an approach that uses nonexpert workers to evaluate small portions of levels generated by an off-the-shelf generation system for the game of Infinite Mario Bros . Several such evaluated portions are then combined to form full levels of the game using a mathematical progression arc model. The composition of the small portions into full levels is done by accounting for the human-annotated information. We evaluated the approach using computational metrics as well as surveying human subjects playing the levels. The results show that the human computation approach is able to generate levels that are perceived by people to have better visual aesthetics and to be more enjoyable to play than existing approaches. Another contribution of our paper is a dataset of the small annotated levels that can be used in future research for learning models for evaluating machine-generated content.