Evaluating The Impact of Dimensionality Reduction Techniques for Decision Tree Performance in Multiclass Imbalanced Datasets

Imbalanced datasets are a common challenge in real-world applications, where the class of interest is often a minority. Addressing class imbalance in multi-class datasets receives less attention compared to binary datasets due to the increased complexity. This complexity arises from varying class frequencies and associated costs. High-dimensional datasets, with numerous features, pose another challenge in machine learning. Feature selection techniques help mitigate dimensionality, improving classifier efficiency in accuracy and computation. These techniques involve creating new features or selecting subsets from the original set. Effective strategies for imbalanced learning aim to retain minority class concepts by leveraging informative features. This study investigates the impact of dimensionality reduction techniques, such as Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA), on multi-class imbalanced datasets using decision trees, a challenge commonly encountered in high-dimensional domains such as bioinformatics and medical diagnostics. While datasets with clear class boundaries may reduce the effectiveness of dimensionality reduction, PCA could be more effective in cases of class overlap, where the majority class has more samples. Experimental results support these conclusions.