Artist-based Classification via Deep Learning with Multi-scale Weighted Pooling

For analyzing digital images of paintings we propose a new approach to categorize them based on artist. Determining the authorship of a painting is challenging because common subjects are illustrated in paintings, and paintings of an artist may not have a unique style. The proposed approach is built upon convolutional neural networks (CNN)---a class of biologically inspired vision model that recently demonstrates near-human performance on several visual recognition tasks. However, training a CNN model requires large scale training data of a fixed input image size (e.g. 224 * 224). In this paper, we propose to construct a multi-layer pyramid from an image, providing 21X more features than using a single layer (i.e., the original image) alone. We train a CNN model for each layer, and propose a new weighted fusion scheme to adaptively combine the decision results. To evaluate the proposed methods, we collect a new painting image dataset, categorized into 13 artists. As demonstrated in the experimental results, the proposed method achieves a promising result---88.08% recall rate in top-2 retrieval on the challenging classification task.