Assessing GPT-4’s diagnostic accuracy with darker skin tones: underperformance and implications

This study aimed to assess the accuracy of GPT-4 in generating appropriate differential diagnoses and arriving at the correct diagnoses for common skin lesions. Additionally, we investigated any differences in its diagnostic accuracy between darker and lighter skin tones. GPT-4 exhibited better performance in providing the correct diagnosis for lighter skin tones (44%, n = 11 of 25) compared with darker skin tones (12%, n = 3 of 25), and this was statistically significant (P < 0.05). Furthermore, with each unit increase in the Fitzpatrick scale, GPT-4’s performance decreased by 11.4% in accurately providing a differential diagnosis and by 7.1% in accurately providing the correct diagnosis.