Double-blinded is not better than “mutually enlightened”: implications of Lord-Besson et al.

In the article by Lord-Besson et al. (1), an important research question was addressed using publicly available data in a creative way. The study compared open-label to blinded clinical trials regarding study discontinuation, overall survival, patient-reported outcome (PRO) completion rates, and PRO change scores. They reported no effect on study discontinuation, overall survival, or PRO completion rates and “no clear direction in differences” or statistically significant differences on PRO change scores. This interesting and well-done study concluded that “concerns about open-label bias should not prohibit the interpretation of large and meaningful treatment effects on PROs.” The authors are to be congratulated on excellent work done to address an interesting and relevant research question. In thinking about the implications of this work for future research, several ideas come to mind. We are quality-of-life researchers who have focused on patient-centered research in general and, more specifically, implementation research and response-shift theory and measurement. Our implementation research has centered on community-based interventions that aim to reach participants where they are and tailor the interventions to participants’ health literacy levels as well as honing engagement-entry points to cultural touchstones. The “Comprehensive Dynamic Trial” design (2) is the quintessential open-label trial, whereby the participant knows what they are getting, and their input helps the intervention to evolve to be more relevant (ie, “mutually enlightened”) (2), culturally responsive, and pertinent to the individual’s context. Empirical research using this trial design has documented substantial and enduring behavioral changes in cancer detection and prevention (3). Nonetheless, a frequent concern raised is the unblinded nature of the design. The work by Lord-Besson et al. (1) suggests that open-label designs do not inevitably lead to biased outcomes. These findings will enhance the application of such Comprehensive Dynamic Trials as well as other unblinded but apparently effective interventions, such as the What Matters to You paradigm (4). Our work on response-shift theory and measurement has examined how adaptation to changing health affects responses to PRO measures and the underlying cognitive appraisal processes that shift. These appraisal processes reflect 4 aspects of how people think about quality of life: frame of reference, standards of comparison, experience sampling, and patterns of emphasis (combinatory algorithm) (5,6). Our research on more than 6000 people with a range of diagnoses—cancer, spinal conditions, neurological diseases, and a wide range of rare diseases in addition to general population samples—has documented that these appraisal processes capture important information about context and what matters to people when their health changes. We have found across patient groups and over time that people’s internal standards, values, and conceptualizations of quality-of-life change when they experience changes in health (7). Studying these appraisal processes over time is an important aspect of personalized medicine (8,9). Indeed, several machine-learning studies have documented that of all the clinical and person factors considered in, for example, orthopedic outcomes studies, appraisal processes remain the most predictive (10,11). If one studies treatment efficacy using methods that assume no appraisal modifications over time, one may miss important changes that affect outcomes (12). When we examine the details of the Lord-Besson et al. (1) work, we note that it focuses on interpretation at the aggregate level rather than on individual differences. As response-shift researchers, we observe that the important domains or symptoms that “favor” the open-label trials are also those domains we would expect to be more subject to response-shift effects. We note that the concept of minimal clinically important difference (13,14)—used as a benchmark in comparing PRO change scores by trial design—imposes a single, aggregate standard for what is “clinically important” rather than considering that such a concept likely varies considerably across individuals (15-17). We thus wonder: what does exposure to open-label trials do to people’s expectations? To their ways of evaluating quality of life? And how might those effects lead to differences in interpreting change scores? We have found, for example, that when one integrates response-shift methods in secondary analysis of (blinded) clinical trials data, a stronger treatment effect is unmasked (18,19). Although it was heartening to note that weighting comparisons using propensity scores did not substantively alter the findings of Lord-Besson et al. (1), we wonder whether there are changes in appraisal processes related to the knowledge that one is in an open-label trial. For example, some people might become more sensitized (over-attend) to some domains or symptoms, whereas others might be so enthusiastic about wanting and getting access to a new promising drug that they understate symptoms that arise in the treatment experience. In accordance with response-shift theory, we recommend that the concept of “bias” itself needs to be reexamined. Bias dismisses the differences as artifactual or something that is somehow not meaningful. The idea of bias here is in fact something that might be quite informative to know how different people’s understanding of their health, treatment, and well-being changes as a function of the (un)availability of information. Rather than viewing any deviation between individuals or groups as “bias,” we might instead think of it from a more patient-centered and phenomenological perspective (20). Supplementing this important foundational research of Lord-Besson et al. with more individual-level study of cognitive appraisal processes would be worthwhile. Characterizing the ways that people think about the experimental treatment they are receiving—whether in open or blinded trials—will increase the relevance of PROs for evaluating personalized medicine. In summary, the Lord-Besson et al. (1) study lays important groundwork for supporting the unbiased interpretation of open-label trials in aggregate studies of cancer therapy. This work supports other important cancer research, such as community-based interventions, which not only are open-label but also invite feedback that leads to the contextualization and evolution of the interventions. From our perspective, studying the cognitive appraisal processes underlying open-label vs blinded trials would be an important next step in this worthy line of research. The existence of validated and accessible measures of cognitive appraisal (21) facilitates this important next step. No new data were generated or analyzed for this editorial. Carolyn Emily Schwartz, ScD (conceptualization; writing – original draft; writing – review and editing); Bruce D. Rapkin, PhD (conceptualization; writing – original draft; writing – review and editing). No funding was used for this editorial. The authors have no disclosures.