The Riddle of Protein Diagnostics: Future Bleak or Bright?

In theory, there is no difference between theory and practice. In practice, there is. The theory behind an exploding kaleidoscope of protein biomarker research is that reproducible molecular differences that correlate with important medical attributes can be discovered in patient samples and that these molecules can be measured with clinical tests to obtain useful medical information. How could this not be true? Chorionic gonadotropin and cardiac troponins are very good biomarkers indeed—for pregnancy and myocardial infarction, respectively—so proteins obviously can serve as useful clinical tests. Apparently the US Food and Drug Administration (FDA) agrees, because, through 2009, it has cleared commercial clinical tests for the measurement of 109 proteins in plasma or serum. Reference laboratories offer “home brew” tests that measure 96 more, for a total of 205 proteins (10% of the proteins known to exist in plasma and 1% of the baseline human proteome) with some recognized clinical importance (1). Of course many, perhaps most, of the chronic diseases that are really difficult to manage, as well as almost all cancers, remain beyond the reach of current clinical laboratory tests. To address this need, the NIH, guided by prevailing theory and the scientific community, has devoted increasing resources to the search for biomarkers via tried-and-true research support mechanisms. Over the past 2 decades, the numbers of research grants, research dollars, and published reports self-identified as biomarker related have increased dramatically (2). Fig. 1 shows the astonishing growth in funding and publication, which we obtained with updated key-word database searches (RePORT) (3). Although we do not know how much of this growth represents real effort directed at clinical biomarkers and how much is cosmetic “repurposing” of other lines of research, the aggregate total of >$1 billion spent over the last 3 years on biomarker discovery ought to have had a major positive impact. Enormous strides have been made in the detection and identification of proteins, largely because of rapid improvements in mass spectrometry instrumentation and software, which are the foundation of modern proteomics. With these tools, very large numbers of candidate biomarkers have been tabulated in publications, amounting in aggregate to perhaps 25% of all human proteins (4). Unfortunately, the impact on clinical diagnostics thus far has been dismal. If anything, the rate at which new tests for protein analytes are cleared by the FDA has slightly declined over the past 15 years, and it seems stuck at <1.5 new tests per year (Fig. 2). This rate is clearly insufficient to address major unmet medical needs and reflects no improvement attributable to the advances made in biomarker research. The tests that have been cleared in this period are not the product of proteomics research and in many cases are neither very widely used nor widely recognized by clinical chemists and physicians. A survey conducted among the leadership in the clinical chemistry community [n = 378, 69 respondents (18%)] revealed that 35% of the tests cleared by the FDA since 1993 are performed in <10% of the clinical laboratories surveyed; 60% of these tests were performed in <25% of laboratories. Furthermore, 50% of the respondents were not familiar with the clinical utility of 12 (45%) of these tests. Although the percentage of respondents in the survey is relatively small, the findings suggest that FDA clearance does not necessarily translate into clinical acceptance, further clouding the impact of the novel markers. HER-2, human epidermal growth factor receptor 2; TnT, troponin T; IL-2R, interleukin-2 receptor; IGFBP-3, insulin-like growth factor–binding protein 3; IGFBP-1, insulin-like growth factor–binding protein 1; sMesothelin, soluble mesothelin-related peptide; sTfR, soluble transferrin receptor; Lp(a), lipoprotein(a); MPO, myeloperoxidase; PCT, procalcitonin; HE4, cancer antigen, human epididymis protein 4; ST2, interleukin-1 receptor-like 1; CKfrag21-1, cytokeratin fragments 21-1; IGF-II, insulin-like growth factor 2; BNP, small form of N-terminal pro–brain natriuretic peptide (introduced in 2002); LP-PLA2; lipoprotein-associated phospholipase A2; TnI, cardiac troponin I; CA15-3, cancer antigen 15-3 (same as CA27.29); Panc amylase, pancreatic amylase. Biomarker theory thus seems to have diverged from clinical practice to an alarming degree. Such a discrepancy in science usually points to a serious flaw in our understanding of the problem at hand and prompts a reexamination of our general approach. At this point, we can identify 2 major structural problems in the approach to protein biomarkers that has been pursued to date. In biomarker discovery as it has been practiced, clinical questions typically have not been very well defined and often have not targeted critical decision points. Distinguishing cancer patients, or other very sick people, from controls is apparently easy—it has been the subject of thousands of papers. Far more difficult, however, is distinguishing rapidly growing from slowly growing prostate tumors (for example), which would provide new information that would profoundly affect treatment. If one assumes that the important clinical questions can be rigorously formulated, obtaining relevant clinical samples is often difficult. Given the effort involved in sample acquisition, the costs of storage and distribution, and the limited sample volumes available, it is not surprising that the institutions and laboratories that manage most sample collections are cautious about sharing their samples. What is surprising is the difficulty of locating these valuable samples. There seems to be no general overview or index of clinical-sample collections that describes sizes, statistical breakdowns, collection methods, and available patient data. Instead, identifying sample sources appears to be largely a matter of who you know. Consequently, biomarker discovery researchers find themselves using “samples of convenience” of unknown integrity. This practice has proved very problematic for some discovery methods, in that unsuspected protein degradation and/or chemical modification occurring during suboptimal storage conditions can lead to false-positive biomarker discovery. To address these issues, we recommend that the NIH develop a list of important clinical diagnostic questions in a variety of disease areas that have been prioritized by healthcare impact and that this list also indicate the corresponding medical centers or consortium studies that can provide the appropriate specimens, along with all needed contact information. These overviews of samples and clinical questions would allow a much more informed and productive application of the existing biomarker resources and would provide a much-needed basis for arguing for the enormous potential health-economic value of successful new tests. Most of the effort and money devoted to protein biomarkers has been focused on “discovery,” almost always with small sample sets. This approach relies on the assumption that the disease-related changes being investigated are large compared with other sources of variation, including biological (population) and analytical (preanalytical and measurement) variance, and thus represent a fairly invariant response across a range of patients. Although the analytical aspect has begun to improve substantially in recent years (5), the primary limitation of biomarker studies to date has been the failure to test biomarker candidates in meaningfully large and well-described patient populations (a stage referred to as “verification”) before declaring success. Very little has been said publicly about the size of studies required to achieve clinical confidence in biomarker work, but the practical rule of thumb Werner Zolg (prior head of proteomics at Roche Diagnostics) proposed in a lecture at the NIH seems about right: If we cannot collect and analyze approximately 1500 samples to test a candidate biomarker, we will have trouble convincing the diagnostics industry, clinical chemists, or physicians that we have found something useful. This so-called Zolg number (Z) would require 6 months of continuous analytical effort with typical “discovery” proteomics analytical platforms, whereas a complete biomarker study to this milestone would require an estimated 4 years and cost approximately $4 million (6). Only a handful of the tens of the thousands of published reports on protein biomarkers have included ≥Z samples. This means that the biomarker pipeline is blocked at its midpoint, and that explains why we have observed only a tiny trickle of new tests emerging from a backlog of thousands of publications. The solution to this problem is clear: Resources must be shifted from pure biomarker discovery toward studies that include definitive clinical verification. This solution will require biomarker researchers to do a great deal more work, in close collaboration with clinical chemists and physicians and often with tools different from those appropriate to unbiased biomarker discovery, i.e., specific assays capable of accurately measuring selected candidates at high precision and high throughput. Furthermore, journal editors should demand much clearer demonstrations of the validity of any presumed novel biomarker. Both of the major problems we have described revolve around an understanding of clinical populations and how to move from limited observations (and biomarker theory) to real people with all their variations (i.e., practice). Can we funnel available resources to the clinical questions of highest value? Can we find (or generate) the samples for investigating these questions? Can we make enough good measurements to answer them? These are fairly practical questions, many aspects of which are naturally addressed through the clinical laboratory.