Usefulness and Effect of Online Prognostic Calculators

To the Editor: Guidelines recommend incorporating mortality risk into clinical decision-making for older adults, such as cancer screening.1, 2 A systematic review of prognostic indices to estimate mortality risk for older adults was previously published.1 To help clinicians, the indices in that systematic review were converted into an online repository of prognostic calculators (www.ePrognosis.org). This study aims to evaluate which online prognostic calculators are used in real-world settings, whether they affect clinical decision-making or the perception of prognosis, and whether they are found to be useful. Usage data collected between April 11, 2012, and April 11, 2013, were examined. Users selected a prognostic index based on the location of the individual (hospital, home, nursing home), time frame of interest for the clinical decision at hand (<2, 2–3, >3 years), and quality of the index (moderate, good, very good, excellent). (The main landing page changed in August 2013; the old landing page is viewable under “Bubbleview.”) Once an individual prognostic index was selected, users were asked whether they were a healthcare professional. If users clicked “no,” a disclaimer explained that the tool was intended for healthcare professionals, and the results should be discussed with a healthcare professional. The user then entered prognostic risk factor data about an individual, such as age, disability, and comorbid conditions. The user was asked to provide his or her best guess of the mortality risk, and then the calculated mortality risk was displayed. Next, the user was asked: “Now that you have seen this information, what is your best guess of mortality risk?” The user was then asked: “Did you find this information useful?” and “Did the information affect a clinical decision?” Finally, the user had the option to click “print the report,” “e-mail the report,” or “finish.” Users who clicked any of the three options had data saved in a completed visit data set, which was used for the present study. Data collected were free of personally identifiable information and could not be linked to individual users. The committee on human research at the University of California at San Francisco reviewed this study protocol and determined that it was not human subjects research. Between April 11, 2012, and April 11, 2013, there were 4,426 completed visits (Table 1), 86% by healthcare professionals. Overall, 63% of indices used were those for community-dwelling residents. Non-healthcare professionals were more likely to use the community-dwelling indices than healthcare professionals (81% vs 60%, P < .001) and less likely to use the nursing home (13% vs 20%, P < .001) and hospital-based indices (9% vs 18%, P < .001). A similar proportion of users’ mortality risk guess agreed with the calculated risk estimate (33%), was more pessimistic (30%), or was more optimistic (38%). When the user was shown the calculated risk estimate, 38% of those whose guess was more optimistic or pessimistic changed their guess to agree with the calculated estimate. Ninety-one percent felt that the calculator was useful. Forty-seven percent of healthcare professionals reported that the calculated prognosis affected clinical decision-making. A limitation of this study was the lack of inclusion of partial users who did not click one of the three options at the end of each calculator to complete the visit. Of completed visits, most users of the online prognostic calculators were healthcare professionals interested in community-dwelling indices. A substantial minority of users changed their mortality risk guess based on the risk estimate that the prognostic calculators generated. Most of the healthcare professionals found the prognostic calculators useful. Conflict of Interest: The editor in chief has reviewed the conflict of interest checklist provided by the authors and has determined that the authors have no financial or any other kind of personal conflicts with this paper. Dr. Smith's effort on this project was supported through the Beeson Career Development Award from the National Institute on Aging (K23AG040772). Author Contributions: McClymont, Smith: study concept. McClymont: drafting the manuscript. Miao: statistical analysis. Smith, Lee, Widera, Schonberg: critical revisions to the manuscript. As the corresponding author, Ms. McClymont had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Sponsor's Role: The study sponsors had no role in the design and conduct of the study; in the collection, management, analysis, or interpretation of data; or in the preparation, review, or approval of the manuscript.