Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems

Several researchers have argued that a machine learning system's should be defined in relation to a specific agent or task: we not ask if the system is interpretable, but to whom is it interpretable. describe a model intended to help answer this question, by identifying roles that agents can fulfill in relation to the machine learning. We illustrate the use of our model in a variety of scenarios, exploring an agent's role influences its goals, and the implications for defining. Finally, we make suggestions for how our model could be to interpretability researchers, system developers, and regulatory auditing machine learning systems.