Answers are grounded in this paper and cite their sources.
O hateful Error, Melancholy’s child, Why dost thou show to the apt thoughts of men The things that are not? Stem cells have captured the attention and imagination of the public, appearing to be a “magic bullet,” sexy and omnipotent, capable of curing health concerns from cancer and cardiac disease to warts and wrinkles. While the scientific enthusiasm for the potential therapeutic impact of stem cells seems justified, the great hope currently placed on stem cell therapies makes the public vulnerable to emotional manipulation, deceptive marketing, or even outright fraud. Concerned by the cacophony of marketing for the so-called stem cell facelift, stem cell breast augmentation, and other “stem cell” procedures, the leadership of the American Society for Aesthetic Plastic Surgery (ASAPS) and American Society of Plastic Surgeons (ASPS) established the Joint Stem Cell and Fat Grafting Task Force in 2010. The task force was charged with gathering information about current stem cell procedures and marketing claims, monitoring stem cell training courses, initiating a dialogue with the Food and Drug Administration, and developing collaborations with key stem cell scientists within and outside the plastic surgery field. In addition, the task force was directed to perform a systematic assessment of the world’s peer-reviewed literature on the clinical uses of stem cells in aesthetic procedures. The “ASAPS/ASPS Position Statement on Stem Cells and Fat Grafting” resulted from that systematic review and critical analysis.1 While the stem cell future may be bright, there is currently scant peer-reviewed evidence to substantiate marketing claims for stem cell aesthetic procedures. What little information is present is mostly of lower levels of evidence (LOE). But why do we consider LOE in the first place, and what purpose do they serve? What does “lower” really mean, why might such evidence be inadequate, and what kind of evidence is necessary before new techniques or technologies—such as aesthetic stem cell procedures—can be confidently adopted? Organized plastic surgery has embarked on a strategic initiative to incorporate principles of evidence-based medicine into all aspects of education, publication, training, certification, and practice.2,3 The resulting evidence-based plastic surgery culture promises to transform both our practices and the process by which we assess—and subsequently accept or reject—innovations such as stem cells. At the same time, if we can effectively communicate the evidence message to the public, we can make significant progress in realigning a dysfunctional plastic surgery marketplace. Currently, this marketplace is characterized by unsubstantiated (or even fraudulent) claims being made far too often and with apparent impunity. Patients—and, unfortunately, physicians—are barraged with pseudoscientific “evidence” that a device or procedure is “medically proven.” Marketing promotions frequently contain the “big four” cognitive errors (or deceptive techniques, depending on your level of cynicism): “cherry-picking” the data, referencing nonexistent data, blurring observation versus effect, and extrapolating clinical outcomes from in vitro experiments (“lab bench to the glossies”).4 When the actual validity of evidence does not really matter, companies have little incentive to invest in substantive clinical research, thus negatively affecting the development and introduction process for new technologies and procedures. In an evidence-valued world, however, careful and methodical research and development would be rewarded, and procedures and technologies supported by solid evidence would prevail (Figure 1). Clearly communicated and readily interpreted, the LOE rankings are a useful tool during this transformation, both in the assessment of an individual study and as a valid metric of the overall literature as we strive to develop a greater compendium of high-level evidence. Pathways to product and procedure development. When evidence does not have high influence in the marketplace, rapid capital procurement and short-term return are primary considerations, while testing and validation are secondary considerations. (A) In this unhealthy system, there is little or no evidence developed (or value placed on evidence) to sustain a product line or prevent disruption by similar nonvalidated concepts; thus, investment in evidence is disincentivized, and product cycles are short. (B) In a healthy system, concepts supported by evidence are rewarded by a competitive advantage in the marketplace and with less chance of disruption by unsubstantiated marketing claims; therefore, investing and reinvesting in a concept supported by valid evidence becomes a sound strategy and product cycles should lengthen. Several systems describe LOE.5 As part of the specialty-wide evidence-based medicine strategy, ASAPS has elected to adopt the ASPS rating scales (Table 1).6 Like most systems, this one uses an ordinal categorization (1–5). Unfortunately, such ranking inherently implies the superiority of “higher” levels (ie, 1–3), while “lower” levels appear relegated to the back of the room, ready for summer school. However, this is not the case. A further exploration of the meaning of LOE shows us that while we do need significantly more high-level evidence studies, lower LOE are not something to be discarded and discredited. They remain a critical part of the innovation and art of plastic surgery. American Society of Plastic Surgeons’ Scales for Rating Levels of Evidence and Grading Recommendations: Evidence Rating Scale for Therapeutic Studies. Adapted from Swanson et al.6 American Society of Plastic Surgeons’ Scales for Rating Levels of Evidence and Grading Recommendations: Evidence Rating Scale for Therapeutic Studies. Adapted from Swanson et al.6 LOE are all about bias. Biases are evolved mental strategies for decision making—cognitive shortcuts, if you will—that may have served well as our ancestors ran from a charging mastodon or pondered which berries wouldn’t kill the kids. Bias is deeply innate, subconscious, and ubiquitous within the human experience. Bias cannot be “turned off” by force of will; it distorts and influences every assessment and decision that we make. While there are many types and classifications of bias, our past experiences, training, emotions, and other factors influence how bias is expressed on an individual basis. Because bias is linked with emotion, bias is like Captain Kirk—rash, emotional, and passionate—sometimes shooting from the hip but always firm in his beliefs. Kirk was pretty good at science, but if you wanted a really, really valid scientific response to your stem cell question, you (and everybody else) would go ask Spock. Spock ran the Enterprise equivalent to the logic-teaching course at the academy, was on every logic panel, and in short was the “go-to guy” on all things logic based. Why? Because Spock could turn off what little emotion he possessed. Emotion leads to bias, and bias distorts logic. If we wish to have the most valid conclusions, we must use clear logic and therefore take steps to limit the impact of emotion and bias. This limitation of bias is the perpetual battle of science, and to succeed, we must understand bias, respect its power, recognize its forms, and defend against its intrusion. Failing this, we—like the mythological Sisyphus—will be constantly rolling our stone of evidence up the hill of validity only to have bias pull it down to the bottom of self-delusion. While bias in ubiquitous, we do have tools that we can employee to limit its effect. Among the most important of these tools are randomization, blinding, chronological perspective, and comparative potential. We can fill our evidence toolbox with various combinations of these, and the summation becomes the study design. The more tools you have, the bigger the job you can tackle, with the greatest chance of limiting bias. An alternative depiction of the LOE helps us visualize how the various tools relate to different study designs and the strengths and points of bias vulnerability (Table 2). Alternate Depiction of the Levels of Evidence. Alternate Depiction of the Levels of Evidence. Randomization alludes to the process of attempting to reduce bias within the selection process. Randomization of necessity is performed after a potential patient has met predetermined eligibility criteria and consented to accept any treatment arm but before treatment is initiated. Failing to randomize appropriately may mean that the outcomes and conclusions are invalid. For example, if sicker, heavier, or more difficult patients are disproportionately assigned to a placebo arm, the treatment arm might appear to be more beneficial than it actually is. Blinding is a process to reduce bias during evaluation and measurement of outcomes by hiding information on the specific treatment from the treating physician (or evaluators), the patient, or both (double-blind). Clearly, direct knowledge of the provided treatment can influence an individual’s assessment of the outcome. Perhaps more subtle, however, even unspoken interactions can affect perceptions, so if either the patient or the evaluator knows which treatment arm was administered, one’s enthusiasm, hope, or other emotions may be unconsciously conveyed to the other and so influence the “blinded” evaluation. Chronological perspective relates to the temporal relationship among decision making, intervention, evaluation, and assembly of data. Because our brains have a profound ability to mold our memory to fit our desires, expectations, and current perspectives, retrospective studies are subject to a spectrum of biases that do not exist in prospective studies. We tend to interpret the past in a way that is biased by our present concepts and attitudes, or as stated by Daniel Gilbert, “when brains plug holes in their conceptualization of yesterday . . . they tend to use a material called today.”7 Finally, comparative potential is the capacity to make direct comparisons between two or more treatment options. The more direct the comparison, the greater the likelihood that variations in skills, biases, periods, systems, patient population, selection processes, and other variables are minimized. By randomizing patients, we are in effect creating the highest possible comparative potential, so controlled randomized trials (Level 1 and 2) have the lowest risk of this type of bias, while in observational, nonrandomized studies (Level 3), the comparison is more distant and the risk of bias increased. Of course, not every tool is applicable to every study situation (for instance, when randomization or blinding is either impossible or unethical), but by becoming more aware of the different types of biases and understanding the relevant tools at our disposal, we can take additional steps to eliminate bias within our research, our critical assessment of the literature, and our practices. While LOE do relate to bias, they do not necessarily relate to study quality or impact. Instead, different LOE fulfill different purposes and answer different questions. Higher LOE, because they are comparative, allow us to understand and measure the differences between treatment options or levels of effectiveness. For example, a Level 1 or Level 2 prospective randomized double-blind trial is best—with best meaning less bias risk and greater likelihood of validity—at determining if one type of pedicle works better than another in breast reduction or for calculating the risk reduction from chemoprophylaxis in a certain group of patients undergoing abdominoplasty. Studies with higher LOE can answer questions like “Is this treatment is better than that treatment?” or “How much more effective is this treatment than a placebo?” No study is perfect, and bias can never be 100% excluded, so even higher LOE studies can have flaws. Problems in randomization, a lack of blinding, faulty outcomes metrics, or inadequate study power (ie, the study is not large enough to accurately predict differences in outcomes) can call the conclusions of even “high evidence level” studies into question. Although lower LOE are more susceptible to bias, studies with lower LOE do serve a critical role as the vehicle for innovation and as a voice of personal experience, both of which are necessary and valuable. On occasion, the lowest level (Level 5) can have profound influence on a specialty, as evidenced by the case reports of Paul Tessier, which lead to the establishment of an entire new subspecialty,8,9 or, potentially, by the recent publication of the first confirmed case of a paraplegic standing and walking with assistance.10 In addition, large case series (Level 4) can identify areas of promise (ie, highly successful treatments) or concern (ie, high complication rates) for further evaluation. Lower LOE show that something is possible, or they convey experience. In a way, while higher LOE studies answer complex questions (“Which is better?), lower LOE articles make simple statements (“This can be done.”). Rather than envisioning the LOE rankings as good or bad, think of the levels as rungs of a ladder. To start, we can’t jump to the top of the ladder (Level 1), so we climb the ladder from the bottom (Level 5) with a new innovation or concept. Gaining experience, we can put together a series of treatments (Level 4) to identify where the promises and pitfalls may lie or to even start a comparison to other groups of patients who have had alternative treatments (Level 3). As we gain adequate experience to standardize the innovation, we can (and should) step up on higher rungs (Level 1 or 2) to make direct side-by-side comparisons. This progression represents the optimal evolution of a procedure alongside concurrent, appropriate (ie, evidence-level) supporting data. The evidence ladder analogy can illuminate two significant problems within the specialty of plastic surgery. The first is a failure to develop the higher-level data in the first place—that is, we accept the status quo, stop climbing up the evidence ladder, and don’t perform the necessary comparative studies to generate definitive clinical answers. By recognizing the need for higher LOE to answer certain critical questions, the specialty has been motivated to surmount this obstacle, although this will require many years and considerable effort to bring to full fruition. The second problem is one of inappropriate interpretation of data according to the LOE. While studies of all levels are critical to a healthy specialty, this serious and pervasive problem arises when lower LOE statements are used to answer higher LOE questions. In essence, this is an extrapolation error (bias) where “lab bench” findings are equated to clinical efficacy or where noncomparative studies (Level 4 or 5) are used to infer clinical superiority. The current rash of stem cell “therapies” seems to demonstrate a high risk for this kind of bias. Although the marketed “stem cell” treatments may vary significantly, in most cases they involve a combination of fat-grafting procedures and some variant of stem cell “activation” and/or “concentration.” Because marketers for stem cells claims clinical superiority over fat grafting alone, the question is not a lower-level issue: “Can this stem cell procedure be done?” Claiming superiority, the marketers have raised the evidence stakes to a high-level question: “Are fat grafts with stem cell enhancement more effective than fat grafts alone?” Such a high LOE question must be answered with high LOE studies. Without side-by-side comparisons, it is impossible to tell whether current “stem cell” procedures are better than, the same as, or worse than fat grafting alone. Since no significant comparative evidence (Level 1–3) currently exists, there is no current scientific basis for aesthetic stem cell procedure claims. Continued research into potential stem cell therapies is warranted and exciting, but lacking such comparative information, the current marketing of stem cell aesthetic procedures is either grounded in ignorance or morally problematic. Ladders are great tools, as long as we don’t try to climb on from the top, get stuck at the bottom, or work from the wrong rung. The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
No takes yet. Share an insight, caveat, or question.
Felmont F. Eaves (2011) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: