Decision tree learning based feature evaluation and selection for image classification

In the big data era, machine learning has become an increasingly popular approach for data processing. Data could be in various forms, such as text, images, audios, videos and signals. The essence of machine learning is to learn any patterns from features of data. In the above types of data, the number of features is massively high, which could result in the presence of a large number of irrelevant features. Most machine learning algorithms are sensitive to irrelevant features so effective evaluation and selection of features in machine learning tasks are highly important. Also, effective evaluation of features can also help identify which features are necessary to be extracted from unstructured data. In this paper, we focus on the processing of image features in classification tasks. In particular, we review two main types of feature selection techniques, namely filter and wrapper. We also review several machine learning approaches that have been used popularly in image classification, and identify the limitations of these algorithms in terms of feature evaluation. An experimental study is reported showing the performance of C4.5 (a decision tree learning algorithm) and other popular algorithms (Naive Bayes, K Nearest Neighbours and Multi-layer Perceptron) on five image data sets from the UCI repository. Furthermore, we describe the nature of decision tree learning algorithms for analysing the capability of such algorithms in terms of feature evaluation in the training stage and for showing how rules extracted a decision tree can be used for evaluating features in the validation stage.