Adapting fisher vectors for histopathology image classification

Histopathology image classification can provide automated support towards cancer diagnosis. In this paper, we present a transfer learning-based approach for histopathology image classification. We first represent the image feature by Fisher Vector (FV) encoding of local features that are extracted using the Convolutional Neural Network (CNN) model pretrained on ImageNet. Next, to better transfer the pretrained model to the histopathology image dataset, we design a new adaptation layer to further transform the FV descriptors for higher discriminative power and classification accuracy. We used the publicly available BreaKHis image dataset for classifying between benign and malignant breast tumors, and obtained improved performance over the state-of-the-art.