A database for perceptual evaluation of image aesthetics

Objective image aesthetics assessment (IAA) is attracting an increasing amount of attention in recent years. One of the most critical issues that hampers IAA research is the lack of publicly available and reliable image databases that can be used to train and test IAA features and models, especially those databases that offer continuous-valued subjective opinion scores. In this work, we construct a Waterloo IAA database containing more than 1,000 images, and carry out a lab-controlled subjective user study. There are several unique and desirable features of the new database as compared to existing ones - It helps us better understand the level of diversity of subject opinions; it provides continuous-valued IAA scores approximately evenly distributed from poor to excellent aesthetics levels; it also allows us to test the effectiveness of various aesthetics features on predicting continuous aesthetics scores. Using the new database as a benchmark, we test more than 1,000 IAA features. The results indicate that existing features are still weak at aesthetics estimation, and the effectiveness of aesthetics features are content dependent. Therefore, understanding and assessing image aesthetics remain a major challenge for future research. The database will be made publicly available.