Common Sense Approaches to Urinary Biomarker Study Design

Since ancient times, physicians have studied the properties of urine to provide clues to the nature of the their patients’ maladies. For example, ancient physicians knew that urine from patients with diabetes is sweet to the taste. The discoveries of such properties were obviously made before disease mechanisms were understood. In modern times, clinicians have added many measurements of urinary composition to provide data for clinical decision-making, relying on growing knowledge of physiology and pathophysiology to identify candidate chemical biomarkers that have been validated for clinical use. Now, aided by methods capable of quantifying thousands of proteins in a urine sample, urinary biomarker discovery has gone “open loop,” allowing the identity of urine proteins that have no known relation to a disease as potential biomarkers for that disease. In such studies, correlation is sought rather than causation. Although this open-loop approach has great promise for accelerated urine biomarker discovery, the lack of connection to pathophysiologic knowledge is a significant limitation. The shift to an open-loop (“unbiased”) approach has been driven largely by the development of modern mass spectrometry techniques that are capable of identifying and quantifying proteins and peptides in large numbers.1 Indeed, recent studies from various laboratories identified more than 1000 proteins present in various biochemical fractions of urine, including urinary exosomes (for a listing of urinary proteins, go to “Urinary Exosome Protein Database” via any internet search engine).2–6 At this point, organizations such the Human Kidney and Urine Proteome Project7 and the European Network for Urine and Kidney Proteomics8 have assembled largely for the purpose of promoting efforts in urinary protein biomarker discovery, and articles addressing various aspects of urinary biomarker discovery have been published. It can only be a short time before large-scale studies evaluating the efficacy of new candidate urinary biomarkers in particular clinical situations will be offered for publication. Consequently, a timely question is, “What should be the ground rules for such studies?” or, more specifically, “How should urinary protein biomarker studies be designed, carried out, and published?” In this brief commentary, I avoid any claim of definitive answers to these questions. Such answers need to be provided by the full community; however, I offer some common sense ideas on how to address these questions. BUILD ON EXISTING THINKING IN THE AREA OF BLOOD PLASMA BIOMARKERS General principles regarding the conduct of protein biomarker studies have been described largely in the context of biomarker studies in blood plasma. For example, a working group at the National Heart, Lung, and Blood Institute (which included two nephrologists) published a general strategy consisting of three phases: Candidate biomarker discovery, biomarker validation, and implementation of biomarker assays (Figure 1).9 This general scheme has been adapted for urinary proteomics.10 Briefly, discovery is conceived as involving large-scale quantification of urinary proteins in a few very-well-defined clinical cases and an equal number of appropriate controls. The output is a list of candidate biomarkers. Validation is a clinical trial involving many more patients than the few enrolled in discovery studies, using appropriate assays for each of the candidate biomarkers. The expected output is a statement of the likely specificity and sensitivity of the assays with regard to the clinical decision-making task being addressed. The implementation phase involves assay development, regulatory approvals (Food and Drug Administration and otherwise), manufacturing, and marketing. Implementation is probably best carried out in the private sector. START WITH THE CLINICAL OBJECTIVE Before engaging in a urinary protein biomarker study, one should document a clear need for a particular biomarker in some aspect of clinical decision-making. It is not self-evident that the availability of a urinary protein biomarker for any given clinical entity will bring benefit. Thus, biomarker development should be driven by needs defined by clinical investigators rather than by the availability of the technology for carrying out protein mass spectrometry. What types of urinary biomarkers might prove useful? Urinary protein biomarker assays can potentially aid diagnosis, allow assessment of prognosis, guide the choice of therapeutic regimen, or provide feedback about response to therapy. An example is identification of protein biomarkers that would help with therapeutic decisions in the setting of a sudden rise in serum creatinine concentration in patients with a renal allograft. Hypothetically, such a rise could be due to graft rejection or tubular injury. Urinary protein biomarker assays that distinguish these two entities could allow interventions while the clinician is waiting for biopsy results and could potentially provide information complementary to biopsy data. The practicality of any hypothetical urine protein biomarker can be addressed a priori on the basis of calculations using likely sensitivity and specificity measures. A proposed project to find a single urinary protein biomarker for use in the general population for early diagnosis of renal clear cell carcinoma, for example, might prove a priori impractical if a specificity of 99% is anticipated (1% false-positive rate). A 1% false-positive rate applied to the general population of the United States would yield millions of false-positive results on absolute terms, engendering unacceptable levels of cost and anxiety. Instead, a battery of markers, each with a limited specificity, may be required to get the overall high level of specificity needed. Therefore, the aim of discovery studies in patients with clear cell carcinoma would be to identify multiple candidate biomarkers to be used in tandem. CHOOSE THE DISCOVERY APPROACH CAREFULLY Protein mass spectrometry and associated protein separation techniques are undergoing rapid evolution and improvement. Therefore, techniques that were state of the art 3 yr ago may be substandard today. Newer instruments are capable of much better mass resolution and therefore greater certainty in protein identification, thereby reducing false-positive results. It will pay, therefore, for clinical investigators planning urinary biomarker development studies to collaborate with leaders in the field of protein mass spectrometry to ensure use of the best possible technology. Explicit identification of specific proteins should be achievable with contemporary mass spectrometers, and it should not be necessary to settle for identification of nondescript mass-to-charge ratio peaks of unknown proteins. THINK ABOUT POSTTRANSLATIONAL MODIFICATIONS AND PROTEOLYTIC FRAGMENTS In biomarker identification studies, most emphasis has been placed on detecting differences in the total abundance of particular proteins in biologic fluids from two populations; however, other aspects of protein regulation, such as posttranslational modifications or regulated proteolysis, may correlate with disease state and provide critical information in the absence of changes in absolute abundances of candidate biomarker proteins. A proposed biomarker for vasopressin action in the kidney is the water channel aquaporin 2 (AQP2).11 Several studies reporting altered AQP2 excretion in various water balance disorders have been published. Yet, in some cases, urinary AQP2 excretion rate does not correlate with vasopressin action,12 and there are important theoretical reasons to expect that AQP2 abundance in urine will not correlate with antidiuretic mechanisms.13 A better approach may be to measure in urine samples the proportion of AQP2 that is phosphorylated at serine-256.5 Serine-256 phosphorylation of AQP2 is mediated by protein kinase A in collecting duct cells in response to vasopressin-evoked increases in cAMP levels and is assessed by immunoassay using a phospho-specific antibody. A particularly desirable aspect of this approach is the possibility of normalization by total AQP2, allowing this measurement to be carried out in spot urine samples. Measurement of other posttranslational modifications, such as ubiquitination, may likewise be useful. Finally, unique proteolytic fragments of particular proteins may be of diagnostic value, particularly in identification of biomarkers for various cancers that secrete characteristic proteases.14 VIEW A VALIDATION STUDY AS A CLINICAL TRIAL As emphasized, validation of candidate biomarkers from discovery studies of urinary protein biomarkers constitutes a clinical trial and as such should be subject to all of the design, regulatory, and review considerations appropriate for any clinical trial. It is anticipated that many, if not most, candidate biomarkers that arise from discovery studies will be false-positive results or eventually shown to have very limited predictive value. It is important to have efficient means of finding the “diamonds in the rough” through the conduct of sufficiently powered validation trials. Ultimately, well-designed validation studies will be what make or break the emerging field of urinary protein biomarkers discovery. RECOGNIZE THE DIFFERENCE BETWEEN VALIDATION OF URINARY BIOMARKERS AND CONFIRMATION OF PROTEIN MASS SPECTROMETRY FINDINGS As discussed, validation of urinary biomarkers is achieved through a clinical trial. The objective of such a clinical trial is to ascertain how well a given protein biomarker or battery of biomarkers predicts the presence of some clinical characteristic important for clinical decision-making. Thus, it should define the likelihood of a false-positive prediction (specificity) and of a false-negative prediction (sensitivity) when a proposed biomarker assay is used in the general population. In contrast, confirmation of discovery results obtained by protein mass spectrometry addresses a different issue: Is the measurement obtained in a mass spectrometry experiment reproducible? A finding may lack reproducibility either for technical reasons related to the mass spectrometry or sample preparation or because of biologic variability. As with any scientific measurement, a single observation or a single set of observations is not enough to draw definitive conclusions. In particular, when hundreds of proteins are observed or measured in individual samples, there are likely to be statistical false-positive results. Thus, any given result should be considered a false-positive result until proved otherwise. What constitutes confirmation under these circumstances? Repeating the same measurement in the same samples helps but not much, because it does not address reproducibility of sample preparation and biologic variability. Repeating the same measurement in new samples from the same individuals is a little better but does not address biologic variability. Repeating the same measurement in samples from a different set of individuals is yet stronger and would allow generalization at a biologic level. Even better would be confirmation of a given finding by a completely different assay method in new samples from new individuals. For example, if mass spectrometry were carried out to discover proteins in urine whose phosphorylation level on a given amino acid was markedly altered in a given clinical state, a strong confirmation would be provided by immunoblotting with a phospho-specific antibody to the detected site using a completely different set of individuals who possess the relevant clinical characteristics. The distinction between validation and confirmation is important because of the issue of how to publish new biomarker studies. If validation is required for publication of discovery studies, then there will be a very long time indeed between the conception of the study and the initial publication simply because of the magnitude of the task of carrying out a clinical trial for validation. However, if discovery studies can be published with confirmation of findings but without formal biomarker validation, then the information would get to the community much more rapidly, allowing laboratories other than the discovery laboratory to benefit from discovery results. Thus, it is important for authors, journals, and reviewers to distinguish between validation and confirmation, and to come to some consensus on the issue of whether full validation is a prerequisite for publication of discovery results or publication should proceed with confirmation only. LOOK FOR BIOLOGICALLY MEANINGFUL MARKERS Protein mass spectrometry has the potential to identify long lists of proteins whose abundances may be altered in urine samples in association with various clinical states. In general, for ultimate success, it makes sense when choosing which candidate biomarkers to confirm or validate to focus on proteins that “fit” with what is known about the biology and pathobiology of the disease under study. Not only do such proteins provide hope of learning about the mechanisms of disease, but also biomarkers that make pathophysiologic sense are more likely to be accepted by the nephrology community. An important constraint in pursuing confirmation or validation is the availability of antibodies that can be used to set up assays for candidate biomarkers in validation experiments. However, modern quantitative mass spectrometry approaches, such as multiple reaction monitoring, are available and allow targeted quantification of selected proteins without the need for antibodies.15 Thus, a lack of good antibodies need not deter investigators from looking at biologically meaningful proteins in confirmation experiments or validation studies. CONCLUSIONS Protein mass spectrometry is undergoing rapid evolution and improvement, allowing large-scale identification and quantification of numerous proteins in individual biologic samples. Publications in which authors have used mass spectrometry for urinary protein biomarker discovery with the aim of developing new clinical tests to aid clinical decision-making have now begun to appear. Such studies represent a marked change in research approach—from sequential, hypothesis-based, mechanistically oriented studies of disease processes to an open-loop approach in which prospective biomarkers that have no immediately recognizable connection to contemporary knowledge about pathophysiology may be identified. Thus, large-scale discovery studies offer both advantages and disadvantages relative to traditional scientific approaches. Here I have presented some of the issues involving the design and conduct of urinary protein biomarker studies with some common sense ideas about how to cope with these issues. Of particular importance is the issue of whether full clinical validation of candidate urinary biomarker proteins (a clinical trial) or just confirmation of protein mass spectrometry results should be normative for publication of biomarker discovery studies. A lot rides on how we approach such problems. DISCLOSURES None.Figure 1: Workflow for urinary biomarker development studies.