Communicating Data About the Benefits and Harms of Treatment

BACKGROUND: Despite limited evidence, it is often asserted that natural frequencies (for example, 2 in 1000) are the best way to communicate absolute risks. OBJECTIVE: To compare comprehension of treatment benefit and harm when absolute risks are presented as natural frequencies, percents, or both. DESIGN: Parallel-group randomized trial with central allocation and masking of investigators to group assignment, conducted through an Internet survey in September 2009. (ClinicalTrials.gov registration number: NCT00950014) SETTING: National sample of U.S. adults randomly selected from a professional survey firm's research panel of about 30,000 households. PARTICIPANTS: 2944 adults aged 18 years or older (all with complete follow-up). INTERVENTION: Tables presenting absolute risks in 1 of 5 numeric formats: natural frequency (x in 1000), variable frequency (x in 100, x in 1000, or x in 10,000, as needed to keep the numerator >1), percent, percent plus natural frequency, or percent plus variable frequency. MEASUREMENTS: Comprehension as assessed by 18 questions (primary outcome) and judgment of treatment benefit and harm. RESULTS: The average number of comprehension questions answered correctly was lowest in the variable frequency group and highest in the percent group (13.1 vs. 13.8; difference, 0.7 95% CI, 0.3 to 1.1). The proportion of participants who "passed" the comprehension test (≥13 correct answers) was lowest in the natural and variable frequency groups and highest in the percent group (68% vs. 73%; difference, 5 percentage points CI, 0 to 10 percentage points). The largest format effect was seen for the 2 questions about absolute differences: the proportion correct in the natural frequency versus percent groups was 43% versus 72% (P < 0.001) and 73% versus 87% (P < 0.001). LIMITATION: Even when data were presented in the percent format, one third of participants failed the comprehension test. CONCLUSION: Natural frequencies are not the best format for communicating the absolute benefits and harms of treatment. The more succinct percent format resulted in better comprehension: Comprehension was slightly better overall and notably better for absolute differences. PRIMARY FUNDING SOURCE: Attorney General Consumer and Prescriber Education grant program, the Robert Wood Johnson Pioneer Program, and the National Cancer Institute.