Feature selection (FS) constitutes an indispensable process in medical data analysis, pivotal for mitigating the curse of dimensionality while concurrently augmenting classification performance and fostering model interpretability. This article introduces a novel hybrid framework, the Adaptive Ensemble of Multivariate Filters (AE-MVF), which synergistically integrates Multivariate Mutual Information (MMI) with extreme gradient boosting (XGBoost)-based feature importance scores within a dynamically weighted ensemble architecture. The core of our contribution is the application of the Harris Hawks Optimization (HHO) algorithm to adaptively determine optimal filter weights and feature subset cardinality, a process explicitly designed to navigate the trade-off between maximizing feature relevance and minimizing inter-feature redundancy. The efficacy and robustness of AE-MVF were rigorously evaluated on a diverse suite of 22 benchmark medical datasets, spanning various dimensionalities, sample sizes, and class distributions. Empirical results demonstrate that AE-MVF yields statistically significant improvements over contemporary filter- and wrapper-based FS methodologies, establishing a new performance benchmark in both classification accuracy and feature subset parsimony. Notably, on high-dimensional, low-sample-size (HDLSS) datasets such as ‘Leukemia’ and ‘Colon’, the framework achieved substantial dimensionality reduction while preserving or enhancing predictive accuracy. In lower-dimensional contexts, AE-MVF delivered competitive performance with highly parsimonious feature sets, often comprising just 3–5 variables. These findings underscore the scalability and generalizability of the AE-MVF framework, positioning it as a potent tool for developing interpretable and computationally efficient models in real-world medical diagnostics.
Al-Adwan et al. (2026) studied this question.