While artificial intelligence (AI) has demonstrated great potential in medicine, humans are ultimately responsible for many diagnostic decisions. To examine how AI can better assist humans in making medical decisions, we conduct an experiment where doctors can see the AI’s diagnoses and are given opportunities to revise their own decisions. We find that complementing AI’s diagnosis with explanations can help doctors improve their performance and mitigate diagnostic expectation – the tendency to overweight the probability of a positive case. Additionally, doctors experience choice overload –they are more likely to follow the AI’s suggestion when shown 3 explanatory features from the AI’s model rather than 10 or 30 explanatory features. We also show that inexperienced and impatient doctors rely more on AI’s assistance. Overall, our findings contribute to the design of better AI-assisted diagnosis systems.
Li et al. (Thu,) studied this question.