Wrapper-based feature selection: how important is the wrapped classifier?

Wrapper-based feature (subset) selection is a frequently used approach for dataset dimensionality reduction, especially when dealing with classification problems. The choice of wrapper is at the forefront of these approaches, whilst the choice of the classifier is typically based on its simplicity as to reduce the computational cost. Since the search is guided by the selected classifier, the same one is also later used for independent testing. This raises the question of how well such feature subsets are suited for other types of classifiers. In other words, can one classifier be used for finding feature subsets that are also effective for others? An investigation into this matter was performed by testing and analysing the utility of subsets found by one classifier with respect to other classifiers. It hints at the importance of classifier choice since some models, whilst used inside the wrapper, can solely conform the dataset to themselves, whilst others are less susceptible to this issue. Consequently, an insight into the robustness of the employed classifiers was gained as well.