Reporting accuracy of rare event classifiers

The article by Rajkomar et al, “Scalable and accurate deep learning with electronic health records” published on 8 May 2018 in npj Digital Medicine , 1 describes an effort to automate the process of taking all the data in an EHR (Electronic Health Record) system, including free-text notes, and transforming it into a format that can then be fed into a deep learning algorithm for various predictions. The authors claim that the deep learning algorithm was able to predict 24-h mortality with an area under the receiver operating characteristic curve (AUROC) of 95%.