Developing the Patient-Reported Outcomes Measurement Information System (PROMIS)

In a process involving hundreds of scientists in 2002, the National Institutes of Health (NIH) identified high-priority scientific opportunities and needs the agency should pursue. Criteria used to identify the most important initiatives included: goals that could not be accomplished by a single institute, but were the responsibility of the NIH as a whole; initiatives that the NIH could not afford not to do; initiatives that no other entity could or would do; and initiatives that would transform biomedical research. This “Roadmap for Medical Research” included, as 1 of 3 major theme areas, the goal of “re-engineering the clinical research enterprise.” Initiatives developed under this roadmap theme are intended to facilitate and enhance clinical research in numerous ways, including promoting integration of clinical research networks, harmonizing regulatory processes, revolutionizing clinical research training, and developing technologies to improve clinical outcomes assessment. This supplemental issue of Medical Care describes the last of these initiatives, the Patient-Reported Outcomes Measurement Information System (PROMIS)—its rationale, the funded network, and the network’s conceptual, analytic, empirical work, and goals. One clinical research issue highlighted by the roadmap process was the need for more valid, reliable, and generalizable measures of clinical outcomes that are important to patients. Conventional measures of disease status do not fully capture the ways that chronic diseases and their treatment affect individuals. Many aspects of patients’ subjective experience, such as symptom severity and frequency, emotional and social well-being, and perceived level of health and functional ability, are important targets for disease intervention. Measurement of patient-reported outcomes (PROs) is particularly important in clinical trials, where laboratory or imaging results may not translate into important benefit to patients, or in trials where 2 treatments may be comparably effective, but have different adverse-effect profiles that differentially affect symptoms, functioning, or other aspects of patients’ quality of life. The identified need for improved PRO measurement engendered an NIH request for applications to develop a validated, dynamic system to measure PROs efficiently in study participants with a wide range of chronic diseases and demographic characteristics. The intent was to create a collaborative group of funded investigators that would take advantage of computer technologies and advances in modern measurement theory to develop an improved tool for measuring PROs. The broad objectives outlined in the request for applications were to: (1) develop and test a large bank of items measuring PROs; (2) create a computerized adaptive testing system that will allow for efficient, psychometrically robust assessment of PROs in clinical research on a wide range of chronic diseases; and (3) create a publicly-available system that can be added to and modified periodically, and that will allow researchers to access a common item repository and a computerized adaptive test (CAT). The first article in this supplement, by Cella et al, provides an overview of the structure and function of PROMIS and summarizes the network’s activity during its first 2 years.1 The authors describe the processes involved in making decisions, directing and monitoring network activity, and standardizing network efforts. As their first major task, PROMIS investigators developed a conceptual framework from which to identify (and select for measurement) outcomes important to patients. The authors discuss in detail the conceptual, qualitative, and quantitative steps taken to define domains and generate item banks for PROMIS. The authors use a case example—the emotional distress domain—to illustrate important aspects of the item bank development process. Having made initial decisions regarding a patient-reported outcomes domain framework and the specific domains for which to develop new instruments, the PROMIS investigators began the process of developing item pools for evaluation and testing. At this phase, carefully-considered psychometric design and analysis decisions became central to the PROMIS process. The next 4 articles address some of the qualitative and quantitative issues in detail, and all 3 quantitatively-focused papers highlight the importance of testing model assumptions. The article by Reeve and the PROMIS psychometric team provides an overview of the network’s design and analysis decisions and their rationales.2 The authors describe a variety of techniques they will use to determine which items will be included in the PROMIS item banks, including both traditional and modern psychometric methods. The final item sets will be calibrated and used to develop algorithms for computerized adaptive testing. These algorithms will determine presentation of the items most informative to estimate an individual’s status on the selected domains. PROMIS investigators approached development of item pools for testing using several different methods. Hays et al describe secondary analyses of Medical Outcome Study data conducted to test the assumptions underlying application of item response models, to evaluate fit of item response models to the observed data, and to evaluate item-level performance, with the goal of informing development of the PROMIS physical function item bank.3 The authors analyzed 15 physical functioning/disability items covering a range of function. The analyses produced information useful in the selection and rewriting of PROMIS test items, and raised issues regarding using aggregated physical functioning scores versus separate scores for different subdomains (eg, lower extremity, upper extremity). Developing PRO scales for children presents some unique challenges. Although PROMIS, as a network, is not developing pediatric PRO item banks at this time, a subset of PROMIS investigators are using a portion of their grant funding for smaller-scale pediatric item bank development, in parallel with the larger PROMIS effort. To inform this process, Hill et al used categorical confirmatory factor analysis to test item response theory (IRT) model assumptions and applied IRT techniques in item analysis and scale evaluation, using pediatric quality-of-life data collected from a large cohort of children and adolescents.4 The authors use this detailed example to demonstrate how IRT analyses can be used to evaluate existing scales or inform construction of new scales. In addition to secondary data analyses, qualitative item review has helped guide PROMIS item selection, modification, and construction. The 6-phase qualitative item review process used by the network is described by DeWalt et al.5 The authors summarize the thinking and methods comprising the network’s rigorous approach, involving extensive collection and review of existing instruments, soliciting patient input via focus groups and cognitive interviewing, revision, and detailed consideration of the choices to be made in all aspects of item construction and presentation. Individuals with disabilities present another set of unique challenges for PRO assessment. The University of Washington PROMIS investigators are providing an invaluable contribution to the PROMIS effort by ensuring that accessibility issues are considered and tested throughout the stages of PROMIS development. Accessibility limitations often greatly proscribe or preclude participation in clinical research for disabled people. The PROMIS PRO domains are clearly major concerns across a wide range of disabling conditions; any CAT tool intended for broad use in chronic disease research must be designed to maximize accessibility. Harniss et al summarize pertinent accessibility laws and regulations, and discuss the universal design principles driving PROMIS activities to ensure development of CAT interfaces and modifications that will meet the needs of disabled patients.6 As the initial NIH Principal Science Officer for the PROMIS Network, I, along with my NIH colleagues on the PROMIS Project Team, collaborated with the PROMIS scientists to form a highly cooperative working network and to develop a scientifically rigorous and transparent process. We have been dedicated to the goals of producing measurement tools that will offer advantages over traditional approaches, of maximizing their usefulness to clinical researchers, and of developing a dynamic, sustainable, resource for both research and clinical application. The goals are ambitious, the work large in scale and broad in scope, and the process and dynamics are complex. As the second year of the Network’s activities drew to a close, I accepted the offer of a position outside NIH, and left with regret my direct involvement in this exciting enterprise. Now, as a clinical research scientist in a nonprofit organization, I see even more clearly the urgent need for the anticipated products of this roadmap initiative, and eagerly await the day when I and my colleagues can access PROMIS tools to enhance the quality and efficiency of our research involving clinical outcomes that are most important to patients.