Commutability Assessment of External Quality Assessment Materials with the Difference in Bias Approach: Are Acceptance Criteria Based on Medical Requirements too Strict?

To the Editor: Commutability of External Quality Assessment (EQA)1 materials is a key requirement for their use in accuracy-based EQA surveys (1–3). In a recent paper, Korzun et al. (4) evaluated commutability of 4 frozen pools for measurements of direct HDL cholesterol (HDLC) and LDL cholesterol (LDLC). These pools were used in the CDC's Lipid Standardization Program to assess accuracy of direct HDLC measurements only (4). Among the results presented using the medical requirement acceptance criteria for bias (4% for LDLC and 5% for HDLC), the authors found that 1 of the 4 frozen pools was commutable for most of the HDLC methods, whereas none were commutable for LDLC methods. The authors concluded that frozen pools prepared according to the CLSI C37 protocol may not always be commutable and especially for direct LDLC assays. In 2013, Laboratoire national de métrologie et d'essais (LNE) organized a similar study to assess commutability of 5 freshly prepared frozen serum pools prepared according to the CLSI C37-A protocol for HDLC and LDLC. The pools were shipped frozen and analyzed along with 20–25 fresh clinical specimens by 31 medical laboratories operating HDLC and LDLC routine methods on the most popular clinical chemistry analyzers: Roche Cobas, Siemens Vista, Abbott Architect, Ortho CD Vitros, Beckman DxC, Beckman AU, Roche Modular, and Thermo Konelab. Commutability results of 5 frozen serum pools prepared according to CLSI C37-A for HDLC and LDLC.a The same 20 fresh clinical specimens (CS) were shipped to 7 laboratories and 12 different sets of 25 clinical specimens were shipped to 12 pairs of other medical laboratories. The fresh clinical specimens and the 5 frozen pools were measured in the same analytical sequence. C stands for commutable, I for inconclusive, and NC for noncommutable, which were all set as described by Korzun et al. (4). Commutability results were obtained using 2 different acceptance criteria based on random error components and medical requirements. The mean of absolute values of the difference for the reference material (DRM) was calculated as defined by Korzun et al. (4) from 9 pairwise comparisons between field methods for LDLC and 32 pairwise comparisons for HDLC. NA, not applicable. To convert concentrations from mg/dL to mmol/L, multiply by 0.02586. Commutability results of 5 frozen serum pools prepared according to CLSI C37-A for HDLC and LDLC.a The same 20 fresh clinical specimens (CS) were shipped to 7 laboratories and 12 different sets of 25 clinical specimens were shipped to 12 pairs of other medical laboratories. The fresh clinical specimens and the 5 frozen pools were measured in the same analytical sequence. C stands for commutable, I for inconclusive, and NC for noncommutable, which were all set as described by Korzun et al. (4). Commutability results were obtained using 2 different acceptance criteria based on random error components and medical requirements. The mean of absolute values of the difference for the reference material (DRM) was calculated as defined by Korzun et al. (4) from 9 pairwise comparisons between field methods for LDLC and 32 pairwise comparisons for HDLC. NA, not applicable. To convert concentrations from mg/dL to mmol/L, multiply by 0.02586. One difference between the 2 studies is that Korzun et al. used the beta-quantification reference method as a comparison method, whereas in our study, pairwise comparisons exclusively involved routine methods. This could affect the outcome of the statistical analysis because sample-specific effects can affect routine and reference methods differently and thus will not be estimated in a comparable manner. Since the measurement procedures included in a commutability study must have similar selectivity for the measurand, we speculate that most materials in the Korzun et al. study were found noncommutable because field methods and the beta-quantification reference method have different specificities. In addition to our use of fewer clinical specimens (20–25 instead of 175), another difference between the 2 studies was that Korzun et al. measured the frozen pools in duplicate at the beginning and end of each run, whereas we performed triplicate measurements only one time in a single run, which did not allow us to consider position and run effects. Our statistical analysis approach was the same as that in the Korzun paper. Consistent with the results of Korzun et al., we found that the criteria for commutability based on random error components (approximately 10%–11%) were approximately 2 times higher than those based on medical requirements (approximately 4%–5%). Although preferable, the application of commutability criteria based on medical requirements appears quite stringent. When examined as if they were a pool, only 23%–27% of the fresh clinical specimens (commutable by definition) were found commutable using criteria based on medical requirements (against 83%–87% using criteria based on random error components), which suggests that medical-based criteria are probably too stringent. The homogeneous methods have been reported to be influenced by specimen specific effects owing to nonspecificity that likely contributes to this observation (5). At the same time, the acceptance criteria based on random error components varied for different pairwise comparisons, making it difficult to define generally applicable criteria. Since acceptance criteria sometimes exceeded 17% for some method pairs, these criteria may not always be stringent enough to validate commutability of materials used as trueness controls. We suggest developing fixed criteria that are appropriate for the intended medical use and that take the performance characteristics of procedures in use into account. External Quality Assessment HDL cholesterol LDL cholesterol Laboratoire national de métrologie et d'essais.