People change their decisions when an AI system contradicts them, given a counterfactual explanation

This is the authors' abstract. We don't add key points for this paper.

We tested whether people change their decisions when an AI system proposes they do so, and the effects of a counterfactual explanation. In eight experiments with 2,369 participants, they made a decision and an AI system recommended they make the opposite one. Participants did not tend to switch their decisions when an AI system proposed they do so, compared to a human expert (Experiment 1), regardless of how it was described (Experiment 1b). Their reluctance occurred whether or not they committed to a decision first (Experiments 2a and 2b). They switched to the AI system’s decision more when it provided a counterfactual explanation than none (Experiment 3); and compared to a causal or evidence explanation (Experiment 4). Explanations did not increase switches to a decision proposed by a human expert (Experiments 5a and 5b). The results occurred whether the AI system proposed a riskier or a safer decision. People are reluctant to adopt a decision proposed by an AI system when it is contrary to their preferred one; counterfactual explanations increase their willingness to do so, because they enable them to consider alternatives. The data are available at https://osf.io/frhk9/.