Answers are grounded in this paper and cite their sources.
BACKGROUND: Previously, we compared six better-developed fatigue patient-reported outcome measurement instruments administered concurrently to a multiple sclerosis sample. Examinations at the instrument level (items) and group level (sample mean fatigue severity scores) implied that all six instruments measured essentially the same variable. However, these same six instruments generated significantly different fatigue severity scores for up to 38% of individuals, implying that they frequently measured different fatigue-related variables. We explored the reproducibility, causes and extrapolation of those seemingly contradictory findings, using patient-reported outcome measures data simulation studies to enable examinations under controlled conditions that cannot be achieved easily by clinical studies. OBJECTIVES: Four simulation studies were conducted to determine whether our clinical findings were reproducible (Study 1), explainable (Study 2), and extendable to measurement of change (Studies 3 and 4). Specifically, Study 1 replicated the conditions of our clinical study, reported in Part 1, to determine whether the findings were reproduced. Study 2 examined how patient-reported outcome measures instrument item number affected the group-level and individual-level equivalence of fatigue scores. Study 3 compared the ability of the six instruments used in our clinical study to detect change. Study 4 examined how item number influences the magnitude and significance of change measured at the group and individual levels. RESULTS: Study 1 reproduced our clinical observations. The six fatigue patient-reported outcome measure instruments produced equivalent mean fatigue severity scores for the sample, but at the same time variable proportions of individuals with significantly different fatigue severity scores. Study 2 showed that patient-reported outcome measure instruments with more items had more individuals with significantly different fatigue severity scores. Study 3 showed that these findings extended to measuring change. That is, group-level analyses implied that the six instruments had the same ability to detect change, whereas individual-level analyses showed that the six instruments had different abilities to detect change. Study 4 showed that instruments with more items detected more people with significant changes. CONCLUSIONS: When comparing patient-reported outcome measure instruments, group-level comparisons alone can be misleading because they can hide statistically significant and clinically meaningful differences between individuals' scores. The extent of these individual-level differences in scores is determined by the patient-reported outcome measure instrument's item number. Importantly for clinical trials, patient-reported outcome measure instruments with fewer items detect less change. These differences are only detectable using modern psychometric methods. Trialists should consider item numbers when selecting patient-reported outcome measure instruments; routinely examine individual differences; and use modern psychometric methods. These findings have significant implications for trial design and analyses.
No takes yet. Share an insight, caveat, or question.
Hobart et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: