Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction

We present a randomized controlled trial for a model-in-the-loop regression, with the goal of measuring the extent to which (1) good explanations of predictions increase human accuracy, and (2) faulty explanations decrease trust in the model. We study explanations based on visual saliency in an-based age prediction task for which humans and learned models are capable but not highly proficient and frequently disagree. Our design separates model quality from explanation quality, and makes possible to compare treatments involving a variety of explanations of levels of quality. We find that presenting model predictions improves accuracy. However, visual explanations of various kinds fail to alter human accuracy or trust in the model - regardless of explanations characterize an accurate model, an inaccurate one, or are randomly and independently of the input image. These findings suggest need for greater evaluation of explanations in downstream decision making, better design-based tools for presenting explanations to users, and approaches for generating explanations.