Pitfalls of Converting Practice Guidelines Into Quality Measures

The Department of Veterans Affairs (VA) manages the largest health care system in the United States, and the Institute of Medicine has recommended that many practices of VA quality measurement be applied to the US health care system as a whole. The VA measures quality of care at all of its sites by assessing adherence rates to performance measures, which generally are derived from evidence-based practice guidelines. Higher adherence rates are used as evidence of better quality of care. However, there are problems with converting practice guidelines, intended to offer guidance to clinicians, into performance measures that are meant to identify poor-quality care. We suggest a more balanced perspective on the use of performance measures to define quality by delineating conceptual problems with the conversion of practice guidelines into quality measures. Focusing on colorectal cancer screening, we use a case study at 1 VA facility to illustrate pitfalls that can cause adherence rates to guideline-based performance measures to be poor indicators of the quality of cancer screening. Pitfalls identified included (1) not properly considering illness severity of the sample population audited for adherence to screening, (2) not distinguishing screening from diagnostic procedures when setting achievable target screening rates, and (3) not accounting for patient preferences or clinician judgment when scoring performance measures. For many patients with severe comorbid illnesses or strong preferences against screening, the risks of colorectal cancer screening outweigh the benefits, and the decision to not screen may reflect good quality of care. Performance measures require more thoughtful specification and interpretation to avoid defining high testing rates as good quality of care regardless of who received the test, why it was performed, or whether the patient wanted it.