Apples to oranges: comparing classification of drone propellers by machine learning models using a standard pipeline

Selection of the appropriate machine learning model is one of the most important elements of data analytics. Too often machine learners are selected based on the research team's previous experience rather than previous successful application or appropriateness of the learner to the problem's domain. To explore how the selection of the learner impacts the feature engineering, this paper takes a previously reported acoustic dataset of four small unmanned aircraft system propellers and applies five machine learners to a feature selection methodology to obtain an ordered list of acoustic features. This analysis demonstrates that each learner requires a different set of acoustic features to obtain a specific level of accuracy. The number of features required for a prescribed accuracy differ across the learners, and application of one learner's list of features did not reproduce similar accuracy when applied to other learners. This suggests that blind application of machine learners to determine salient features should be checked against multiple analyses and domain knowledge to validate the selection criteria.