Moderate agreement for cardiac output monitors

This Invited Commentary accompanies the following original article: Slagt C, Helmi M, Malagon I, Groeneveld ABJ. Calibrated versus uncalibrated arterial pressure waveform analysis in monitoring cardiac output with transpulmonary thermodilution in patients with severe sepsis and septic shock. An observational study. Eur J Anaesthesiol 2015; 32:5–12. The past decade and a half has seen an explosion in the number of devices and techniques that monitor cardiac output. A frequent question for the anaesthesiologist and intensivist is therefore ‘what do the numbers from this new monitor tell me about my patient?’ One way we might answer this question is by comparison with data from established techniques. If a new and less invasive monitor reliably gives the same data as a familiar piece of equipment that has our trust, then it is easy for us to incorporate it into our practice. In this issue of the Journal, Slagt et al.1 compare calibrated (VolumeView/EV1000, COap) and uncalibrated (FloTrac/Vigileo, COfv) pulse contour methods of measuring cardiac output with a thermodilution technique (COtptd). Thermodilution has been around in one form or another for over 40 years2 and no doubt features in many of our own personal diagnostic strategies. Slagt et al.1 studied septic patients, a group whose often profound vasodilation has previously proved challenging for cardiac output monitors.3 COap, the calibrated technique, gave a bias of 0.02 l min−1 and limits of agreement of −2.49 to 2.52 l min−1 when compared with COtptd. Unsurprisingly, the uncalibrated technique fared worse with a bias of −0.86 l min−1 and limits of agreement of −4.48 to 2.77 l min−1. Bias itself is not necessarily a huge problem provided that it does not vary with cardiac output. Inspection of the Bland–Altman plots in Figure 1 suggests that it does not,1 so we can easily ‘correct’ the COap or COfv result by adding the mean bias. Scatter, as measured by the limits of agreement, is more of an issue, as the wider the region the ‘true’ result may lie in, the less confidence we can have in the numbers displayed on the monitor. How precise do we need our monitor to be? Traditionally, studies of cardiac output monitors have taken the approach suggested by Critchley and Critchley4 who proposed limits of agreement of ±28.3%, rounded to ±30%, as the threshold for reporting acceptable agreement between two monitors. Their approach is based on combining the variances of two methods of measurement, each with assumed limits of precision of up to ±20%. However, this is not so much a statistical question as a clinical one. How much does the value for a cardiac output have to change before our working diagnosis or our treatment strategy has to change as well? In other words, what degree of variation are we prepared to ignore as insignificant at the bedside? The use of an agreement:tolerability index (ATI)5 is a way of incorporating this question into the analysis. The ATI is a ratio of the limits of agreement from a study of two monitors to a predefined region of practical equivalence, to use a Bayesian term,6 or the range of cardiac outputs we would accept without changing our decision making. This is also defined as a tolerability interval that separates extremes of classification.5 The smaller this ratio, the more likely it is that the results from the two monitors are sufficiently close to prompt us to pursue the same treatment strategy regardless of which monitor we had looked at. An ATI of less than 1 has been proposed as demonstrating acceptable, 1 to 2 as marginal and greater than 2 as an unacceptable agreement.5 The ATI is sensitive to the predefined range of values for the tolerability interval. Slagt et al.1 suggest an ATI of 1.2 for the comparison between COap and COtptd and an ATI of 1.8 for COfv and COtptd, demonstrating the superiority of the calibrated technique but showing both pulse contour monitors as having marginal agreement with transpulmonary thermodilution. They use a generous interval for equivalent values of 4 l min−1, based on the reference range for cardiac output of 4 to 8 l min−1. Many intensivists would intervene, or at least review the situation, if the cardiac output unexpectedly dropped by 3.9 l min−1. On the contrary, a change in cardiac output of 3.9 l min−1 would not necessarily misclassify a patient from having cardiogenic to distributive shock or vice versa. Slagt et al.1 describe the agreement between the various monitors they test as moderate. Whether this is moderately good or moderately bad probably depends on how one uses the data from the cardiac output monitor. As an example, taking the baseline data in Table 2 for the uncalibrated COfv, the limits of agreement can be crudely estimated at a range of 9.5 l min−1. This represents an error of approximately±60%, an ATI of 2.4 and therefore unacceptable agreement for the baseline data alone! So, although the authors report ‘moderate agreement’ overall for both monitors, it may also be interpreted that with an ATI of 1.2 for the calibrated COap, it is moderately good, but for COfv at 1.8, it is moderately bad! Acknowledgements relating to this article Assistance with the commentary: none. Financial support and sponsorship: none. Conflicts of interest: none. Comment from the editor: this Invited Commentary was checked and accepted by the editors but was not sent for external peer review. MC is an associate editor of the Eur J Anaesthesiol.