A Novel Ordering Index for Evaluating Feature Selection Quality in Personalized Smart Healthcare

Wearable technology and the Internet of Things have increased access to personal data, enabling applications that deliver individualized treatment and therapy within clinical pathways. To optimize coaching and interventions within such pathways, it is essential to identify all relevant factors in the available data. Feature selection can be a useful tool for achieving this, but with small, high-dimensional datasets, common in healthcare, it can be challenging. The goal of this study is to develop a method for identifying the most relevant features in small, high-dimensional datasets and to introduce a new ordering index that measures the quality of the orderings produced by feature selection methods. This novel index is sensitive to the quality of feature ordering and to the prediction model’s performance metrics when combined with a feature selection method. The index reaches its maximum when the number-of-features-versus-accuracy graph has an ideal concave-downward shape, reflecting increasing accuracy with each informative feature added and decreasing accuracy with each confounding feature added. Using this index, we define six feature orderings derived from the results of four standard feature selection methods. Using the performance metrics and our new ordering index, we show that the resulting orderings can identify more relevant features and improve the overall performance of the classification models, and that the ordering index is a useful contribution to feature selection techniques.