Toward Explainable Data and Sports Analytics: A Case Study on Pass Completion Prediction in American Football

The sports analytics industry has seen a rapid influx of data, driving widespread adoption of machine learning for predictive analytics. However, this shift has often sidelined conceptual understanding and explainability. Focusing on pass completion prediction in American football, we develop transparent models matching black-box performance. In this work, we use pass completion prediction as a case study and build explainable models that achieve comparable performance to black-box approaches when trained on spatio-temporal features. Our approach involves two key strategies: first, developing a conceptual understanding to engineer a small explainable feature set; second, leveraging this understanding to create a physics-based probabilistic model as a single equation. Surprisingly, both logistic regression and the transparent model achieved 72% accuracy, matching complex models. Using the small explainable feature set, an ensemble model reached 78% accuracy, outperforming the state of the art while maintaining interpretability. Beyond prediction, explainable models provide actionable insights into play dynamics, player performance, and training strategies. In contrast, black-box models, while potentially improving prediction accuracy, obscure which features contribute to predictions, how they interact, and why specific outcomes occur. This lack of transparency limits their ability to inform decision-making and guide future investigations.