What Is a High-Quality Randomized Controlled Trial?

Clinical trials play a critical role in decision-making, because they are specifically designed to answer questions regarding outcomes resulting from the application of a specific clinical intervention. A well-designed clinical trial can provide conclusions regarding the intervention, including those about safety, efficacy, and durability of treatment; however, many clinical trials are not as well designed as we would like to believe [1]. They are prone to numerous sources of biases, such as skewed randomization, inappropriate placebo group, and inadequate blinding, as well as subliminal interpretation, including conflicts of interest (COI). Biases, whether by design failure or interpretation, can lead to inaccurate results, precipitating confusion among practitioners, and the potential to abrogate the development of clear treatment guidelines due to conflict between studies [2, 3]. High-quality clinical trials develop a clinically important and ethically appropriate comparison with a priori identification of the inherent biases of study design in order that the results are valid outside of the trial and broadly applicable. The increasing awareness of the abundance of low-quality clinical trials is arguably reflected in the establishment of the CONSORT statement, Cochrane Collaboration, and empirical attenuation of risk of bias over the decades [4]. These tools serve as proactive measures for transparent reporting and study design to minimize risk of bias. Despite these efforts, a high risk of bias in clinical trials persists [2]. Thus, we aim to highlight four critical areas of scientific design and potential bias which reduce the quality of clinical trial research: 1) stratification of known confounders; 2) blinding; 3) placebo response; and 4) COI. The primary goal of a well-designed trial is to have the statistical power to definitively arrive at a conclusion, often referred to as confirmation or rejection of the null hypothesis, that is, that groups are not statistically different [3]. However, this challenge is deeper. Individuals are often inherently different, because of variables such as age, race, sex, or underlying health conditions, and this can lead to variations in their responses to treatment. Therefore, to minimize the risk of substantial baseline differences between groups, study subjects are randomized into treatment or control groups to provide an unbiased distribution of underlying factors aside from the active treatment. This attenuates the potential for what is termed selection bias. However, simple randomization fails to necessarily distribute all factors that have been established to impact outcomes. The failure of randomization is particularly notable in smaller groups or very specific conditions, where small differences may have more profound impact on outcome. For example, in spine surgical studies, many factors of outcome beyond patient demographics have been well identified in large, retrospective analyses, such as severity and type of pathology. Many conditions within a broad diagnosis are simply not comparable; groups need to be balanced for specific confounding differences between patients. Therefore, known predictors of outcome need to be identified a priori during trial design, and randomization integrity is validated either by (1) transparent allocation of confounding variables across treatment and control groups or (2) stratification of patients based on a priori known predictors of outcome and then application of randomization into treatment cohorts. Similarly, following CONSORT guidelines, if transparent allocation is deemed sufficient for study purposes, there should be reporting of the specific variables assessed and demonstration of no significant differences between groups [3]. Application of this study design is crucial for the dissection of whether treatment effects are consistent, with or without bias due to known predictors of outcome. Additionally, stratified study design also serves delineate if certain preexisting conditions may amplify or negate treatment effects. Arguably, studies may choose to not apply such stringency for the sake of simplicity of patient accrual and speed of randomization, because stratification would require increased participant recruitment, time, and resources [2]; however, without implementing risk stratification prior to randomization, the post hoc analysis needs to precisely validate that known predictors did not bias the study. A study that neglects the known factors that contribute to the specific condition evaluated in the trial is therefore a low-quality trial. Since the placebo effect was first reported, the inclusion of the placebo group has revealed that health changes from administration of an “inert” treatment should not be ignored and demonstrates the power of subliminal response to alter clinical course. The need for a placebo group is especially critical in treatment of disease with a substantial psychological constituent, such as pain. Because of the multidimensional and affective nature of pain perception, psychological, computational, and clinical hypotheses exist to explain the improvement of symptoms without associated improvement of the biological underpinnings of the disease [5]. Although placebo groups may fall under ethical scrutiny because of historical research involving “placebo with deception,” there is scant evidence that unethical placebo trials exist in current practice [5]. Furthermore, studies suggest that placebos, when clearly identified and given to patients “honestly prescribed” or “open label,” can affect health outcomes to a greater extent than controls with no additional treatment—strengthening the rationale for ubiquitous application of placebo control groups [5]. Inclusion of an active placebo group or comparison of surgical technique has become the norm to ensure a high-quality clinical surgical trial, because it has been demonstrated that comparison of surgery to medical management is an inaccurate comparison. For example, after high-quality, level one evidence found no benefit of intended surgery versus sham, strong recommendations [6] were made against subacromial decompression [7]. There is compelling evidence for placebo response to sham surgery. A critical component of high-quality clinical trials is blinding (masking), which is intended to decrease subjective confounding biases [2]. Double-blinding is the most stringent study design, where both participants and study staff are unaware of the intervention throughout the study duration [3]. Maintenance of double-blinding can be difficult in the context of surgical intervention, without sham surgery, or drug placebos, because of the recognition of active treatment or side effects. In these studies, at the very least, the study staff measuring outcome variables must be blinded to treatment group, such as masking a surgical incision, especially if the outcome test involves subjective assessment, such as a physical examination. Furthermore, evaluators should be independent to the study, particularly from influence of practitioners with vested interests, including academic prestige and clinical income [8]. Without blinded, independent evaluators, bias may arise due to subliminal influence, even on objective outcomes, such as timed activities, which often involves instruction and repeat testing. Patients tend to improve on objective measurements when encouraged by evaluators, even if no treatment has been given [9]. In guideline development, one of the most frequent causes of demotion of RCTs to level two evidence is the lack of appropriate blinding [10]. COI can occur in many forms, including financial gain, academic prestige or promotion, or personal benefit, such as a family member deriving benefit from effective treatment. Notably, financial COI remains the most convoluted, because most trials are funded by either industry inducement or non-industry grant, each of which can involve pressure for performance. Surveys have revealed interpretation bias or change in study design because of pressure from a funding source [11]. Furthermore, there is often a compounding effect from principal investigators when they serve as consultants and/or equity owners, because they influence both the concept or marketing of a surgical product as well as the design and conduct of the clinical trial. This tension has culminated in support for transparent reporting of industry relationship, yet a positive association between industry-funded clinical trials and more favorable trial outcomes persists throughout the literature [11]. These results are corroborated at the individual level; principal investigators with financial ties were similarly associated with increased reported efficacy of treatment. Thus, despite mechanisms to account for potential biases, industry-funded study results are overwhelmingly influenced towards positive results. Collectively, these issues present an incredible challenge in overcoming bias due to COI. Additionally, it must also be acknowledged that academic trials with a lack of commercial influence are not abdicated from similar COI. One technique has been to submit the design of the trial to peer-review in advance, both for critical evaluation of trial design as well as that negative data cannot be selectively ignored in final publication. Despite these qualifiers, there is a pressing need to critically appraise the results of commercially funded clinical research. Several proposals exist such as enforcing third-party, independent statistical analysis of RCTs or, most revolutionary, restricting participation of those with significant COI [11]. Clinical trials are paramount in providing guidance in clinical treatment. Results have the power to upset prevailing clinical dogma—yet many trial results are downgraded or dismissed in clinical guidelines because of the presence of biases. Through assiduous trial design and ethical reporting of all results, including negative findings, trials become valid, reproducible, and applicable to clinical decisions. Surgical trials remain the most challenged by these concerns, especially by complex pathology-based risk stratification, placebo effects, blinding, and COI from the medical industry. Thus, it will be important to address these areas for improvement, starting with increased critical awareness of the trap of low-quality RCTs. COI = conflict of interest; RCT = randomized controlled trial; CONSORT = Consolidated Standards of Reporting Trials Funding sources: NIH, AANS/CNS NREF, Boston Scientific, and SPR Therapeutics. Disclosures: Consultant for: joimax gmbh, Renerva LLC, SPR Therapeutics. conflict of interest randomized controlled trial Consolidated Standards of Reporting Trials