Error Methods Are More Practical, But Uncertainty Methods May Still Be Preferred

This is the authors' abstract. We don't add key points for this paper.

“Error methods—compared with uncertainty methods—offer simpler, more intuitive and practical procedures for calculating measurement uncertainty and conducting quality assurance in laboratory medicine. However, uncertainty methods are preferred in other fields of sciences as reflected by the guide to the expression of uncertainty in measurement.” (1). Those are the opening sentences in the report from the Total Error Task and Finish Group (TE-TFG)2 of the European Federation of Clinical Chemistry and Laboratory Medicine (1). The TE-TFG was appointed by the European Federation of Clinical Chemistry and Laboratory Medicine as 1 of the outcomes from the 2014 Milan conference on quality specifications (2) to resolve issues regarding total analytical error (TAE) vs measurement uncertainty (MU). The focus of the 2014 Milan meeting was to review existing recommendations for setting analytical performance specifications and to update the guidelines from the 1999 Stockholm Consensus Conference. One outcome was that the hierarchy of goal-setting models was reduced from 5 to 3 classes—specific models for clinical use, general models related to individual and population variation, and “state-of-the-art” recommendations from expert groups and proficiency testing (PT) and external quality assessment (EQA) programs. Although the PT/EQA criteria for acceptable performance fall into the lowest class, these survey programs are well established globally and require goals for TAE because only a single measurement is allowed on survey samples and that measurement is subject to both random and systematic errors, i.e., precision and trueness. Thus, there is an ongoing need for the TAE model and goals for TAE. TAE vs MU became an issue when ISO 15189 was published in 2007 and required that the “uncertainty of results” be considered in the accreditation of a medical laboratory (3). Another ISO document, Guide to Expression of Uncertainty in Measurement (GUM), became the obvious source for satisfying this new requirement (4). GUM described a detailed and complex “bottom-up” approach for determining all the components of uncertainty (as SDs) and then combining them (as variances, SD2) to provide a final estimate of MU (as the square root of the sum of variances). Such an approach is useful for manufacturers to identify sources of errors, make improvements in analytical systems, and characterize traceability. For laboratories, however, the bottom-up GUM methodology is too complicated. To simplify applications, the 2012 revision of ISO 15189 suggested a “top-down” approach that utilizes intermediate-term quality control (QC) data to estimate MU. In the US, where the TAE concept was introduced >40 years ago (5), TAE is well accepted and widely used. Its integration with Six Sigma quality management has led to the development of many practical applications for laboratory quality management. Based on specifying an allowable total error (ATE, TEa) as a “tolerance” criterion for the quality required for intended use, Sigma-metrics can be calculated [(ATE − |Bias|)/SD] to provide a measure of quality. Sigma-metrics may be used for judging the acceptability of a new method based on initial validation studies, guidance in the selection of appropriate statistical QC (SQC) procedures, assessment of quality of PT results, and indicators of risk when developing risk-based QC plans and designing risk-based SQC procedures as set out in Clinical Laboratory Standards Institute (CLSI) guidelines EP23A and C24-Ed4. US laboratories have shown little interest in MU because its determination is not required by the Clinical Laboratory Improvement Amendments Act. However, the issue has been more widely debated globally, particularly in Europe, which has led me to recount the original purpose and argue for the practical usefulness of the TAE approach (6–8). Laboratory professionals in the US need to be aware of these discussions because this issue is not going away and will reappear in the future. Guidance from ISO is influential with manufacturers, who in turn are influential in CLSI, and who have already brought US laboratories risk-based QC plans and risk-based SQC. Fig. 1 illustrates the relationship of the fundamental performance characteristics (trueness, accuracy, precision) and their measures in the error and the uncertainty models. ISO recognizes that accuracy is a function of both precision (SD) and trueness (Bias) when only a single measurement is made (which is common practice in medical laboratories), but it does not recommend TAE as a measure because it is not an accepted term in the International Vocabulary of Metrology. Instead, MU is identified as the measure, under the conditions that (a) bias be eliminated, corrected, or ignored (Bias = 0) and (b) the uncertainty in the estimate of bias (UB) be included along with precision (SD) in the estimated MU. Note that the error model does not commonly include UB, although it was included in the original publication (5), along with the uncertainty in the estimate of the SD (USD), which is generally larger than UB but not commonly included in either model. Thus, there are critical differences between the models, but the models would converge if Bias were zero. UB is the uncertainty in the estimate of bias, and USD is the uncertainty in the estimate of the SD. The handling of bias is the crux of the debate between TAE and MU (9). The TAE model accounts for the bias observed for a method, whereas the MU model assumes that method bias can be eliminated, corrected, or ignored. Proponents of TAE argue that method bias exists in the real world, even for harmonized, standardized, and certified measurement procedures. The hemoglobin A1c test serves as an example for which there is global harmonization (IFCC), national standardization (US, Sweden, Japan), and sometimes national certification of methods (e.g., US National Glycohemoglobin Standardization Program). Despite these efforts, PT/EQA surveys continue to show significant biases between method subgroups, e.g., see the results from a 2016 College of American Pathologists survey for >20 method subgroups (10). A persistent criticism of the TAE model is that the uncertainty in the estimation of bias is not included in the estimation of TAE: “Bias is included in the conventional TAE model as a constant without uncertainty. MU methods take this uncertainty into account. It is composed of the uncertainty of the values of the bias and of the reference standard. To improve TAE estimation, the uncertainty of the bias was suggested to be incorporated into the TAE calculation.” (1). Ignoring the uncertainty in the estimate of the bias is seldom a problem because it is small compared with the estimate of the SD for imprecision and small compared with the uncertainty in the estimate of that SD, which is ignored in the MU methodology. If proper accounting for the uncertainty in the estimate of errors is a strict requirement, both the TAE and MU methodologies are flawed, and the MU methodology more seriously so than the TAE methodology. The major advantage claimed for the MU methodology is that the estimates of uncertainty from different sources can be combined to estimate the uncertainty of a larger process, e.g., the uncertainty of the total testing process could be estimated from the individual uncertainties of the preanalytic, analytic, and postanalytic phases. The most important preanalytic consideration is the individual biologic variability of a patient, which may be large for some analytes, e.g., approximately 6% for cholesterol compared with a typical method CV of 2% to 3%. The TAE model is often criticized because it considers only bias and imprecision and does not include other sources of variation. “This [TAE] model is only valid when imprecision and bias are the only variables involved…. Thus—in its original form—the TE model does not include biologic variation and other causes of variation into the model….” (1). The TAE model can be expanded to a “clinical decision interval model” (11) that accounts for within-subject biologic variation, sampling, or specimen bias, between-specimen sampling variation, averaging of results for the number of tests, specimens, and replicate measurements, plus the precision and bias of the measurement procedure and the uncertainty of detecting unstable performance because of decision rules and number of control measurements chosen for SQC. Most of the discussion about MU in the past 20 years has focused first on the bottom-up GUM methodology and later on the more practical top-down alternatives. However, the interest in the TE-TFG report seems to be shifting from MU to diagnostic uncertainty (12). This perhaps explains the continuing preference for MU concepts and the GUM methodology, as discussed in the concluding paragraph of the TE-TFG report (1): “Analytical performance specifications should take the diagnostic uncertainty of the whole testing process into account. MU methods according to GUM have not been sufficiently developed to deal with diagnostic uncertainty in laboratory medicine. Development of analytical performance specifications for diagnostic uncertainty has the potential of creating a paradigm shift in laboratory medicine resulting in quality improvements and improved use of diagnostic methods. In anticipation of this paradigm shift, error methods remain the most used methods for quality assurance and analytical performance specifications in laboratory medicine. Currently, error and uncertainty methods are complementary when evaluating the measurement results in medical laboratories.” This paradigm shift to diagnostic uncertainty is a lofty and revolutionary goal for the future; however, it must be achieved by the evolution of existing quality management practices that deal with the reality of laboratory medicine and the theory of metrology. The TAE model and its integration in Six Sigma quality management have provided a practical approach with useful tools and techniques for measuring, monitoring, and improving the quality of laboratory testing processes (13). Continuing applications are providing new tools for risk-based quality management (14, 15). For MU, as noted in the TE-TFG report (1), applications for quality management are lacking, and further developments are necessary to define the permissible MU, as well performance specifications for the whole diagnostic organization. In anticipation of these applications, laboratory analysts need to pay attention to the further development of MU because ISO trends and directions will influence practices in the US, as we have already witnessed in the CLSI guidelines related to risk management. Total Error Task and Finish Group total analytical error measurement uncertainty proficiency testing external quality assessment Guide to Expression of Uncertainty in Measurement quality control statistical quality control Clinical Laboratory Standards Institute.