Programmatic hyperparameter optimization (HPO) methods, such as Bayesian optimization and evolutionary algorithms, are known for their sample efficiency in identifying optimal configurations for machine learning (ML) models. However, practitioners often use less efficient methods, such as grid search, potentially resulting in under-optimized models. This discrepancy suggests that HPO method selection may be influenced by practitioner-specific motives, which remain insufficiently understood hindering user-centered advancement of HPO tools. To uncover these motives, we conducted 20 semi-structured interviews and an online survey with 49 ML practitioners. We revealed six primary goals (e.g., increasing ML model understanding) and 14 contextual factors (e.g., available computational resources) that influence practitioners’ choices of HPO methods. This study provides a conceptual foundation for understanding real-world HPO practices and informs the development of more user-centered and context-adaptive HPO tools in automated ML (AutoML).
Kannengiesser et al. (Wed,) studied this question.