Performance of popular pulse oximeters compared with simultaneous arterial oxygen saturation or clinical-grade pulse oximetry: a cross-sectional validation study in intensive care patients

ABSTRACT Background The demand for pulse oximeters is high in the current COVID-19 pandemic. Despite their popularity, clinical studies to evaluate the reliability in obtaining information on a patient’s oxygenation status, are lacking. Aim To evaluate the performance of pulse oximeters under clinical conditions, with arterial blood gas measurement (SaO 2 ) as reference standard. Methods We studied the accuracy of ten top-selling pulse oximeters in Europe and the USA in an intensive care population at the Flevoziekenhuis in Almere, the Netherlands. The studied pulse oximeters were: AFAC FS10D, AGPTEK FS10C, ANAPULSE ANP 100, Cocobear, Contec CMS50D1, HYLOGY MD-H37, Mommed YM101, PRCMISEMED F4PRO, PULOX PO-200, and Zacurate Pro Series 500DL. Adult patients in whom an SaO2 measurement was obtained as part of routine care were asked for inclusion. Directly after obtaining the SaO2 blood sample we obtained the pulse oximeter readings (SpO 2 ) of the investigational devices, in random order. Outcomes were bias (SpO 2 – SaO 2 ) mean, root mean square difference (A RMS ), mean absolute error (MAE), and accuracy in identifying hypoxemia (SaO 2 ≤90%). As a clinical index test, we included a hospital-grade SpO 2 -monitor. Results In 35 consecutive patients we obtained 2,258 SpO 2 -readings and 234 SaO 2 -samples. Mean bias ranged from −0.6 to −4.8. None of the pulse oximeters had an A RMS ≤3%, the requirement set by ISO-standards and required for FDA 501(k) clearance. The MAE ranged from 2.3 to 5.1, and 5 out of 10 pulse oximeters met the requirements of ≤3%. For hypoxemia, negative predictive values were 98-99%. Positive predictive values ranged from 11-30%. Highest accuracy (95%CI) were found for Contec CMS50D1; 91% (86-94) and Zacurate Pro Series 500DL; 90% (85-94). The hospital-grade SpO 2 -monitor had an A RMS of 3.0 and MAE of 1.9, and an accuracy of 95% (91-97%). Conclusion Direct-to-consumer pulse oximeters do not meet ISO-standards required for FDA-clearance, but can accurately rule out hypoxemia.