Answers are grounded in this paper and cite their sources.
Traditional outcomes in observational studies and clinical trials, such as high-contrast visual acuity or formal perimetry, fail to capture the patient's experience of living with neuro-ophthalmic disease. Patient-reported outcomes (PROs) seek to measure the impact of disease and interventions on a patient's psychosocial and physical health without intervening interpretation from clinicians and researchers. Given the recent growth of PROs within clinical research, it is increasingly important for neuro-ophthalmologists to understand the value of PROs and challenges faced when interpreting or designing studies with PROs. Furthermore, PROs, previously used only within the context of clinical research, are now being employed within the electronic health records of health care systems across the United States and globally to examine the quality of care and the patient experience, improve patient–clinician communication, and support shared decision making (1–3). In this companion article to “Long-Term Patient-Reported Outcomes of Visual Field Defects and Compensatory Mechanisms in Patients After Cerebral Hemispherectomy” by Meer et al, we aim to provide a brief primer on PROs for neuro-ophthalmologists, considerations for appraising studies that use PROs, and the basic principles for selecting PROs for research. AN OVERVIEW OF PATIENT-REPORTED OUTCOMES What are Patient-Reported Outcomes? Patient-reported outcomes are any health measures directly conveyed from the patient. PRO measures, also called PRO instruments, are questionnaire-based tools used to measure a health state, symptoms, functioning, or quality of life based on direct report from an individual patient. Ideally, PRO measures are more than just a collection of questions on a particular topic or related to a specific disease. PRO measures are developed around conceptual models in order to understand a concept, such as quality of life (4). In general, 3 types of PRO measures have been used in vision-related research: generic PRO measures that are used to measure quality of life in both ocular and nonocular conditions (e.g., 36-Item Short Form Survey (5)); general vision-related PRO measures used to measure vision-related quality of life and vision functioning (e.g., National Eye Institute Visual Function Questionnaire (NEI-VFQ) (6,7), 10-item Neuro-Ophthalmic Supplement (8)); and disease-specific PRO measures that are designed to measure disease-specific quality of life, symptoms, and functioning (e.g., 15-item Myasthenia Gravis Quality of Life questionnaire [MG QOL-15]) (9). Value of Patient-Reported Outcomes in Clinical Research Patient-reported outcomes complement traditional outcomes and in some cases may provide the most important information to patients and clinicians about the effect of a disease or intervention. Many clinical trials result in no measurable impact on patient care because of failure to select outcomes that are meaningful to patients and clinicians (4). PROs are one potential solution to making clinical trial data relevant. They can provide information to patients and clinicians about how an intervention tested within a clinical trial may impact daily life (10). In fact, PRO measures collected within clinical trials can be used in some countries, including the Unite States, to support regulatory claims that an intervention positively impacts health-related quality of life and functioning (11). In other cases, certain PROs, such as the EQ-5D, can facilitate health economic analyses (12). Comparative effectiveness research is particularly suited to the use of PROs. In comparative effectiveness research, 2 interventions found to have similar efficacy are compared within an observational study or clinical trial. Because traditional measures of treatment effect (e.g., high-contrast visual acuity) are likely to be comparable between treatment arms, PROs can help distinguish differences that matter most to patients, such as impact on vision-related quality of life. PROs can also measure how treatment burden, such as dosing frequency, costs, or other inconveniences, may interfere with quality of life. Challenges of Patient-Reported Outcomes in Clinical Research Although PROs are meant to promote and amplify the patient voice, many challenges exist to using PROs in clinical research. First, there are many PRO measures that have been developed, but their quality is highly variable (13). Development of high-quality PRO measures requires establishing which concepts are important to patients and experts. Additionally, these concepts must be measured reliably by the instrument, the threshold for clinically meaningful change must be established, and the PRO must be able to distinguish a positive or negative change with a degree of precision. Additionally, a PRO should be well-targeted to the population in which it will be employed, which means that its items should represent the entire functional or quality of life spectrum of those who will undergo the assessment. As a result, high-quality PRO measure development can be time consuming. Research teams with expertise in qualitative research methodology, survey design, and psychometric data analysis are essential for designing new PRO measures. Second, most PROs have not been rigorously validated. Even among those that have undergone validation, the quality of validation varies greatly (13). Third, these challenges translate into limited evidence on the impact of PROs on clinical care (14). In a recent systematic review of the impact of PRO data from clinical trials, only 17% led to a real-world change in clinical practice, such as modifications to guideline statements (14). Finally, patients with neuro-ophthalmic diseases often live with visual and physical disabilities necessitating the use of a surrogate (e.g., caregiver, family member) to complete a PRO instrument, as in the study by Meer et al. In this study, they incorporated measures from the PedsQL Pediatric Quality of Life Inventory (15), which has validated parent-proxy versions. However, PRO measures that are not designed to be used by a surrogate and validated accordingly may not accurately represent the views of the patient. Patient-Reported Outcomes in Neuro-Ophthalmology Few validated PRO measures exist that are specific to rare diseases. This is particularly problematic in a discipline such as neuro-ophthalmology, where nearly all the diseases we treat are considered rare. Even when disease-specific PRO measures exist, such as the MG QOL-15 (9) for myasthenia gravis, most were not developed or validated in patients with primarily neuro-ophthalmic manifestations. Therefore, most neuro-ophthalmology clinical trials that have used validated PRO measures, such as optic neuritis trials (16–20) and the Idiopathic Intracranial Hypertension Treatment Trial (21–23), have used general vision-related quality of life and visual function measures. It is important to note that these more general measures may be not be sensitive to the full effect of an intervention, as these instruments were not designed around the needs of the target population and their treatment goals (24). However, the use of more general PRO measures is still valuable and can still provide useful information about concepts important to patients across different diseases. Table 1 provides examples of the common general PRO measures that may be applicable in neuro-ophthalmology research. The Patient-Reported Outcome Quality of Life Instruments Database (PROQOLID) and Patient-Reported Outcomes Measurement System (PROMIS) maintains repositories of PRO measures and item banks (25,26). TABLE 1. - Examples of general patient-reported outcome measures applicable to neuro-ophthalmology research Health-Related Measures Number of Items Time to Administer* Domains/Subscales Additional Information Short Form Survey (SF-36) (5) 36 items 5–10 minutes Physical functioningPhysical role functioningSocial role functioningPhysical painHealth perceptionsVitalitySocial role functioningMental health Widely used across diseases SF-6D is available for economic evaluation studies and to determine quality adjusted life years EuroQoL 5D (EQ-5D) (12) 25 items and a visual analogue scale to indicate general health status Less than 5 minutes MobilitySelf-careUsual activitiesPain/discomfortAnxiety/depression Widely used across diseases Used for economic evaluations Youth and proxy formats are available Health and well-being instrument is available PedsQL Pediatric Quality of Life InventoryGeneric Core Scale (15) 23 items 5 minutes Physical functioningEmotional functioningSocial functioningSchool functioning Widely used in pediatric research Numerous condition-specific modules available, some are neurology specific (epilepsy and brain tumor) Vision-related and neurologic-related measures National Eye Institute Visual Function Quality of Life (NEI-VFQ) (7) 25 items 5–10 minutes General healthGeneral visionOcular painNear activitiesDistance activitiesVision-specific social functioningVision-specific mental healthVision-specific role difficultiesVision-specific dependencyDrivingColor visionPeripheral vision Most commonly used questionnaire to measure vision-related quality of life. Poorly captures impact of impaired contrast sensitivity and ocular motility problemsShorter and longer versions available NEI-VFQ Neuro-Ophthalmology Supplement (8) 10 items Less than 5 minutes Neuro-ophthalmology specific subscale to the NEI-VFQ-25 Designed to be used with the NEI-VFQ-25 Captures double vision, difficulty focusing or following moving objects and difficulty with vision when the eyes are “tired” Impact of Vision Impairment (IVI) (40,41) 28 items 5–10 minutes Reading and accessing informationMobility and independenceEmotional well-being Instruments are available for children and low-vision patients. Brief 15-item instrument is available Item banks and computerized adaptive tests are available Vision-Related Quality-of-Life Instrument for Children and Young People (VQoL_CY) (42) 35 items 15 minutes Unidimensional scale to capture child-perceived psychosocial impact of visual disability Functional Vision Questionnaire for Children and Young People (FVQ_CYP) (43) 36 items Not reported Unidimensional scale to capture child-perceived difficulty performing vision-dependent activities Quality of Life in Neurological Disorders Measurement System (Neuro-QOL) (26) Variable Designed to be completed in 1 minute General and disease-specific measures available covering physical, mental, and social health21 adult self-report domains and 11 self-report pediatric domains available Measures of the functions, symptoms, behaviors, and feelings important to people with neurologic diseases and their caregivers Proxy versions availableItem banks, computer adaptive tests, and short forms available (6–8 items) *Time to administer may be dependent on the specific study sample. EVALUATING CLINICAL RESEARCH STUDIES USING PATIENT-REPORTED OUTCOME MEASURES Assessing Patient-Reported Outcomes Measure Development Three important components to assessing PRO measure development are validity (content and construct validity), reliability, and responsiveness. Content validity describes the evidence that a measured construct, such vision-related social functioning, is important to patients and experts. This is achieved by conducting qualitative research (e.g., interviews, focus groups) with patients and often also with experts, clinicians, and other stakeholders (e.g., caregivers). Another component of content validity is an understanding of how constructs may differ among important subgroups (e.g., by gender or socioeconomic status). These qualitative data are used to develop an evidence-based, conceptual model that specifies key domains and how they relate to the central construct and each other. The qualitative data are also used to develop a set of potential items, often questions, that are tested with patients. The potential items developed from the qualitative data are narrowed to those that best capture a concept of interest, by using direct patient feedback and results of psychometric analyses, as discussed briefly below. Construct validity refers to the degree to which the questions in a PRO instrument measure what they intend to measure. Researchers assess construct validity during the development phase using psychometric methods. Within clinical research, measurement reliability refers not only to the reproducibility of a measurement but also the ability to distinguish between patients despite the presence of error in the measurement tool. This is accomplished typically by comparing the variability that occurs in responses between patients completing a measure and the variability that occurs both between patients and within patients when the measure is repeated. Responsiveness refers to the ability of a PRO measure to detect clinically meaningful differences. Therefore, defining what constitutes clinically meaningful differences in the outcome must be explored during the PRO development phase. Psychometrics is a quantitative discipline that uses mathematical models to understand how the items within a PRO measure relate to the theoretical concepts delineated in the conceptual model. Common psychometric analyses, such as factor analysis or principle component analysis, aim to explain how the specific items within an instrument relate to an idea that cannot be directly observed (e.g., vision-related social functioning). Other common psychometric analyses, such as item response theory (IRT), and particularly Rasch modeling, are used to describe the relationship between an individual's response to an item and the broader trait that the instrument seeks to measure, such as vision-related well-being. Item response theory and Rasch approaches consider the difficulty of each item for scoring and do not assume an equal interval between all response options. For example, if a person is responding to a question about difficulty carrying out a specific task, a change from “somewhat difficult” to “moderately difficult” may not be mathematically or conceptually equivalent to a change from “moderately difficult” to “very difficult.” Because IRT/Rasch-transformed PRO scores are on an interval logit scale, they can be used with standard parametric statistical analyses. Massof has provided a comprehensive summary of the application of IRT/Rasch models to the validation and interpretation of PRO measures (27). Assessing Patient-Reported Outcome Measure Implementation, Analysis, and Interpretation After assessing PRO measurement development, it is important also to consider how a PRO measure was implemented, and how the data were analyzed and interpreted within the study. When assessing PRO measure implementation, the population being studied should reflect the population in which the measure was validated. It is also important to consider if the timing and frequency of PRO measure administration was appropriate. As in all clinical research studies, the primary outcome of interest should be identified. However, PRO instruments may provide multiple scores in a variety of domains; therefore, the primary area of interest should also be identified. With regard to the statistical analysis plan, power assessments in intervention studies using PRO measures have unique considerations. Detecting clinically meaningful differences among high and low performers may be more difficult than in the middle ranges due to floor and ceiling effects, thereby affecting power calculations. Similarly, if scores are skewed in general toward the top or bottom of a scale, this will need to be considered. Because appropriately scored PRO measures result in separate scores for each area assessed (e.g., ocular pain or vision-specific well-being in the NEI-VFQ), the statistical analysis plan needs to consider how multiple outcomes will be addressed. For studies that use repeated PRO measures, appropriate statistical models should be used, such as repeated-measure analysis of varience or generalized estimating equations. Missing data are also quite common among studies using PRO measures, making them particularly prone to response bias if only complete data is analyzed. The presence of missing data within individual respondents (e.g., skipping questions) and over time (e.g., failing to complete a follow-up PRO measure) should be accounted for in the analysis plan. Standard approaches for handling missing data exist, such as imputation, and results can be compared to complete case analyses (28). Importantly, studies should address how the results translate into clinically meaningful change both within individual patients, if multiple time points are collected, and between individual patients. For clinical trials using PRO measures, guidelines exist for reporting protocols (SPIRIT-PRO) (29) and results (CONSORT-PRO Extensions) (30). The Future of Patient-Reported Outcome Measures Traditionally, PRO instruments have been completed in full and may impose significant response burden on study participants. Newer methods of PRO instrument administration involve the use of computerized adaptive testing (CAT). Administration of a PRO instrument using CAT employs a thresh-holding approach to efficiently estimate the trait of interest (e.g., vision-related social functioning), where the number of items administered is not predetermined and is instead based on the patient's responses and the standard error of measurement deemed acceptable by the investigator. On subsequent administrations, testing may begin using prior information from past tests, thereby further improving efficiency. Because CAT can also increase measurement precision, studies that administer PROs using this approach may actually require smaller sample sizes. Several recent publications have highlighted applications of CAT in vision and neurologic research (26,31–34). Selecting Patient-Reported Outcome Measures for Your Research Based on the International Society for Quality of Life Research (ISOQOL) recommendations (35), neuro-ophthalmologists should consider the following when designing their own research studies using PROs. First, consider how content was developed and how the important concepts are measured. The evidence for the reliability, validity, and responsiveness should be documented. Second, if the PRO measure is deployed in a language that differs from the language in which it was designed, it is important to ensure that a rigorous translation process was used and that questions are interpreted as intended. At a minimum, this requires qualitative interviews with potential patients to ensure that the translation is capturing the correct concepts. Third, the data generated should be interpretable and meaningful, not only to the researchers but also to the patients and clinicians meant to benefit from the research. Last, the burden on patients completing the PRO measure and the burden on investigators administering the measure should be considered and minimized. Items should be clear, easy to understand, and written in plain language (36–39). CONCLUSIONS Patient-reported outcomes support patient-centered care by measuring outcomes that matter most to patients. To insure that the care we provide to patients in neuro-ophthalmology has a meaningful impact, it is essential that clinical research in neuro-ophthalmic diseases adopt rigorously developed and validated PRO measures that are appropriate for patients with neuro-ophthalmic conditions. Considerable opportunities exist to improve our treatments and the care that we deliver to patients with neuro-ophthalmic disease, and PROs are one essential tool to help us achieve that aim.
No takes yet. Share an insight, caveat, or question.
Lott et al. (2021) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: