Is overfeat useful for image-based surface defect classification tasks?

One of the challenges for real-world image-based surface defect classification task is the lack of labeled training samples to extract useful features to confidently classify defects. In this paper, we present results on our investigation on whether features derived from OverFeat, a variant of Convolution Neural Network, can be used directly for image-based surface defect classification task. We show that the classification performance of two real-world defect images datasets can be significantly different. For the harder classification task, OverFeat features are useful for some types of surface defects, but performs poorly when the defects demonstrate characteristics beyond texture patterns. We propose a simple heuristic approach called Approximate Surface Roughness (ASR) that provides auxiliary information on the relationship between spatial regions in the defect image to be used together with the OverFeat features. Empirical results show improvement in classification performance for those defect types that do not classify well using only OverFeat features.