Advances in functional magnetic resonance imaging (fMRI) have led to the ability to study the brain across many contexts. However, the large number of features generated by functional connectivity approaches may overfit the data. These problems can be overcome with either feature selection (FS) or dimensionality reduction (DR), which can be applied to less complex models. We utilize two open source datasets to compare the performance of DR/FS methods on cognitive task decoding using a suite of ML classifiers. While FS and DR methods have been used previously in decoding research, no systematic comparison of their performance has been undertaken. Here, we compare available methods using commonly utilized machine learning libraries to establish which methods provide the best predictive performance. We then conduct statistical tests to examine the relative contributions of DR and FS methods and classifiers on decoding accuracy. Neither DR or FS was found to be superior. However, differences were identified across datasets and tasks. In the majority of methods and datasets, a peak in predictive performance was found using a small percentage (005-.10%) of the total number of original features. Some methods perform better than the baseline method of prediction with all available features or selecting features randomly. Decoding performance with some datasets with some method exceeds that of deep learning approaches. These results suggest a “sweet spot” for the tradeoff between the retention of features and predictive accuracy. • The present study compared a suite of dimensionality reduction (DR) and feature selection (FS) methods while using functional connectivity data for cognitive task decoding in two datasets • Neither DR or FS was found to be superior to each other • A “sweet spot” of using.005%-10% of the total number of functional connectivity features was found to be optimal for decoding accuracy • Using ridge regression with a suite of DR/FS methods, decoding performance exceeded that of state-of-the-art deep learning approaches
Richier et al. (Sun,) studied this question.