SKI: An Agile Framework for Data Science

This paper explores data science project management by first noting the need for a new process management framework and then defines a process framework that effectively supports the needs of a data science team. The paper also reports on a pilot study of teams using the framework. The framework adheres to the lean Kanban philosophy but augments Kanban by providing a structured iteration process for teams to incrementally explore and learn via lean hypothesis testing. Specifically, the Structured Kanban Iteration (SKI) framework focuses on having teams define capability-based iterations (as opposed to Kanban-like no iterations or Scrumlike time-based sprints). Furthermore, unlike Kanban, the framework leverages Scrum best practices to define roles, meetings and artifacts. Thus, SKI implements the Kanban process, but with a more repeatable and structured approach.