FACO: A Novel Hybrid Feature Selection Algorithm for High-Dimensional Data Classification

High-dimensional datasets, encompassing rich and valuable information, are frequently encountered in numerous real-world scenarios. However, the task of classifying such datasets using machine learning algorithms is challenging due to the phenomenon known as the “curse of dimensionality.” To address this issue, feature selection emerges as a crucial concept in machine learning. Its primary goal is to identify and select a subset of relevant features from the vast high-dimensional feature space. In this paper, we propose a novel hybrid feature selection algorithm based on a modified ant colony optimization (ACO) and fisher discriminant analysis (FDA). In the ACO, we propose two new rules, which modify heuristic information and the pheromone update equation. Also, we include fisher discriminant as a penalty term in the ACO heuristic information to prevent the algorithm from being stuck at local optima and to improve the ability of the ACO algorithm to select the optimal feature subset, thereby combining both the strength of the filter and global search approach. This feature subset generated by this algorithm is fed into a k-nearest neighbor, and the resulting algorithm is called FACO. We test the performance of FACO on both synthetic and real-world datasets, and the performance is compared with KNN, KNN&SBS, ABACO, BACO, BGA, and BPSO. The results show that FACO outperforms all other algorithms by more than 3% on both synthetic and real-world datasets. The results also show that the hybrid search strategy used in FACO can locate optimal feature subsets in high-dimensienal feature space.