Use of Patient-Reported Outcomes to Improve the Predictive Accuracy of Clinician-Reported Adverse Events

Abundant research has now demonstrated that patient and clinician reports of symptoms—and particularly symptomatic toxicities (ie, adverse events) during cancer treatment—provide discrepant yet complementary data ( 1–3 ). How can this be? Can’t only the patient or the clinician be “right”? The more patient-centered among us might state that the patient is always right by definition because nobody (not even the most sensitive clinician) can truly know another person's subjective experience. But the more traditional among us might assert that clinicians should be considered right because they have an “objective” perspective based on experience and training, which prevents them from exaggerating or understating what they observe. In fact, it appears that both the patient and clinician provide information of value, which when combined provides a more accurate understanding of the patient’s symptoms. This finding is good news for those of us who are interested in improving the measurement of symptoms in clinical trials and practice. The optimistic among us might state that all we need to do now is figure out how to operationalize this approach. The current standard mechanism for reporting toxicities in cancer research is clinician-only reporting using items from the National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE) ( 4 ). Any cancer clinician is familiar with these approximately 800 items—about 10% of which represent symptoms such as nausea, fatigue, or sensory neuropathy—and all of which are currently reported by clinicians only, with no explicit mechanism for patient direct reporting. The article by Quinten et al. ( 5 ) in this issue of the Journal demonstrates that adding information gathered directly from study participants via patient-reported outcome (PRO) measures improves the predictive accuracy of clinician CTCAE reporting (eg, raising the adjusted C-index from 0.63 to 0.67 for fatigue and from 0.62 to 0.65 for nausea, which represents a substantial improvement) ( 6 ). This important finding adds to previous work supporting the added value of PRO measurements to existing mechanisms for clinician toxicity reporting. Previously, we reported in the Journal ( 2 ) that although clinicians are able to predict survival well, patient reporting of symptomatic toxicities better reflects the patient's underlying health state and functional status. Moreover, patient reporting appears to detect potentially serious adverse event symptoms earlier than clinician reporting. This phenomenon was notably demonstrated by the timing and cumulative incidence of patient vs clinican reporting of severe diarrhea in the IFL (irinotecan/fluorouracil/leucovorin) arm of NCI intergroup trial N9741 ( 7 ), in which an abundance of life-threatening gastrointestinal serious adverse events was ultimately detected ( 8 ). Therefore, availability of PRO data not only enhances the accuracy of clinician CTCAE reports but also may improve safety. So, operationally how might this work? There are three potential approaches: 1) “Independent reporting,” in which patient and clinician toxicity data are collected, analyzed, and reported completely separately from each other; 2) “Merged reporting,” in which patient and clinician data are collected separately and then merged analytically into a single metric; and 3) “Collaborative reporting,” in which patients directly report symptomatic toxicity information, which is then provided to clinicians to inform their CTCAE reporting. Of these three approaches, we favor the third, collaborative reporting. This approach provides clinicians with valuable information about the patient experience immediately to inform not only their required research reporting but to support their ongoing management of symptoms and communication with patients. There is ongoing research to determine which approach is most informative (described below). Moreover, the third approach capitalizes on knowledge that the patient and clinician have of each other rather than relying on an artificial calculation to merge their perspectives. (There are also existing models of independent reporting by professionals vs consumers that are well accepted, such as commercial websites that post the scores of professional reviewers side-by-side with aggregated consumer reviews of anything from electronics to movies to cars to hotels.) A collaborative reporting approach was recently integrated into a phase II clinical trial of paclitaxel, pemetrexed, and bevacizumab in patients with advanced non–small cell lung cancer at Memorial Sloan-Kettering Cancer Center (E. Basch, M.C. Pietanza, and M. G. Kris, unpublished data). At each visit, via wireless tablet computers, 44 patients self-reported 13 CTCAE symptoms (alopecia, anorexia, cough, dyspnea, epiphora, epistaxis, fatigue, hoarseness, mucositis, myalgia, nausea, pain, sensory neuropathy) using a previously developed questionnaire (2). This information was transmitted in real time to clinicians’ wireless computers, on which they could either agree or modify the patient-reported grades ( Figure 1 ). Over an average of 13 return visits per patient (range 1–57), patients completed self-reports 99.7% of the time (672/674 total visits). The six participating research nurses agreed with patient-reported grades 93% of the time, raising severity 2% of the time, and lowering it 5% of the time. Screen capture from “collaborative patient–clinician CTCAE reporting” web interface used in a phase II clinical trial. Patients self-report CTCAE symptomatic toxicities in the waiting room and then clinicians either agree or modify the patient-reported grade and assign attribution in real time. CTCAE = Common Terminology Criteria for Adverse Events. Although it is not known how these clinicians’ grades might have differed in the absence of patient self-reported information, presumably there would have been greater disagreement given the many prior publications demonstrating discrepant independent patient–clinician reporting (including the Quinten et al. article in this issue). Therefore, these findings provide preliminary evidence that clinicians, when presented with patients’ self-reports, will modify their own adverse event documentation, making use of the information provided by the patients. A formal comparison of these three approaches is planned as a part of the NCI’s PRO-CTCAE initiative, which is developing a patient-reported outcome version of the CTCAE ( 9 ). This initiative, in which the authors are involved, has developed PRO items for 80 symptomatic toxicities specifically for the purpose of collecting adverse event information in clinical trials, to be complementary to the CTCAE and to other adverse event reporting systems such as the Medical Dictionary for Regulatory Activities (MedDRA). Patient accrual to a large validation study of the PRO-CTCAE items is near completion, and feasibility studies are beginning within multicenter trials in the NCI cooperative groups. Integrating the patient perspective into drug safety reporting will not only improve the accuracy of the data collected but also will enhance the patient-centeredness of clinical research. In oncology, a field in which symptoms are common and can substantially impair patients’ functioning and quality of life, information about the patients’ experience of toxicity is essential for multiple stakeholders, including patients, drug developers, regulators, and payers: Patients facing a treatment decision wish to know what they can expect in terms of symptoms, based on the prior experiences of “patients like them.” Drug developers wish to understand how well patients will tolerate a product. This is particularly relevant with oral therapies for which compliance is strongly associated with symptomatic side effects and can be useful in early-phase research toward identifying tolerated dose levels and in pivotal trials to compare tolerability between products from the patient perspective. Regulators have long recognized the limitations of symptomatic adverse event information reported by clinicians in trials. A systematic patient-reported approach would increase confidence in the fidelity of this information toward balancing risks and benefits. Payers wish to understand how ill patients will feel with particular treatments because it helps to predict the utilization of health-care services. In summary, patients are in the best position to report their subjective experiences, whereas clinicians are in the best position to contextualize that experience in terms of disease continuum. Both bring a valuable perspective that can complementarily inform our understanding of treatment toxicities. Ideally, the next version of the CTCAE (to be version 5), will incorporate into its design the use of patient-reported data to improve the accuracy of its symptom items.