A Survey on Convolutional Neural Networks and Their Performance Limitations in Image Recognition Tasks

Convolutional neural networks (CNNs) have shown outstanding image classification performance, having been successfully applied in several real‐world applications. However, there are performance limitations encountered in CNNs and few works have pointed out these limitations across the literature. Therefore, this survey aims to provide a comprehensive explanation of both the importance and performance limitations of CNNs on image classification tasks. In this paper, we start with a brief CNN background and follow the main CNN architectures along with their contributions and benchmark datasets. We propose a classification of the CNNs performance limitations, which are grouped into four categories: labeled datasets, translation invariance, adversarial attacks, and spatial relationship. In addition, we describe some approaches that are currently being developed to overcome these performance limitations. Finally, we introduce specific ongoing research to address these performance limitations through capsule networks (CapsNets) and CNNs combined with CapsNets.