There is a persistent paradox in the data science domain: despite the growing recognition of data as a strategic asset, many projects designed to leverage its value still suffer from high failure rates. To address this challenge, this study introduces the Data Science Projects Success Assessment Model (DS PRO-S), developed using a Design Science Research approach to make data science project success explicit, measurable, and comparable. DS PRO-S functions as a meta-model and an instantiation toolkit, complete with an operational methodology that supports success and health assessments using critical success factors (CSFs) and success criteria at both the phase and project levels through four distinct modules. This modular structure enables evaluations at any point in the data science lifecycle and informs timely, data-driven interventions before issues propagate. The measurement and evaluation framework within DS PRO-S aligns with ISO/IEC 15939, incorporating mathematical formulations for aggregating success criteria and CSFs into upper-level scores. To demonstrate its instantiability, completeness, and operational utility, case studies were conducted in a predictive analytics project of a large energy enterprise and a generative AI project of a vendor. The findings indicate that DS PRO-S is applicable in diverse project contexts in the data science domain and offers a robust solution for assessments.
Gökay et al. (Fri,) studied this question.