Quality Control and Protein Microarrays

The scale of testing required in clinical analysis has of necessity driven analytical methods to a variety of multiplexed formats. A single assay in a test tube has evolved to automated multiple testing in cuvettes in multicuvette carousels, or wells of a microplate with up to 9600 wells (1). Other formats for multiplex testing include 2-dimensional microarrays of spots of reagents on a planar surface and microbeads either in suspension [so-called liquid or 3-dimensional (3D)1 arrays] or held in wells on the end of a fiber optic bundle (2)(3). The microarray testing format has grown in popularity for both protein and nucleic acid analysis, and is now the dominant method for applications such as gene expression and single nucleotide polymorphism analysis. Multiplex testing presents specific analytical and quality control challenges. One factor that complicated early array-based tests in a microplate format was the regional differences in results due to different thermal properties of the outer vs the inner wells of the plate—the so-called edge effect. It was observed that the thermal properties of a well in the interior of a microplate differed significantly from wells at the edge of the array. The consequent temperature differences affected the results of immunoassays performed in microplates, a problem resolved by better control of incubations (4). In most modern-day microarray-based tests, a variety of performance controls are included to monitor analytical performance and safeguard the quality of the analytical data. Such controls include blank areas, positive controls, negative controls, normalization controls, controls for nonspecific binding and to confirm sample addition, cross-reactivity controls, and controls to assess the activity of assay reagents such as the detection conjugate (5). The report in this issue by Klee and colleagues (6) highlights an underlying concern in microarray-based testing that relates to the quality of multiplexed assays. Their article reports the results of measurements of 15 protein biomarkers in a large study involving 2322 participants from a community-based cohort. The study also included a plasma control and recombinant protein controls. Analysis was performed using 2 custom planar microarrays with 6 or 9 capture antibodies printed in individual wells of microplates (an array of arrays). The study used standardized protocols for sample processing, storage, and freeze-thaw exposures. The results of this study provide an important illustration of the sample processing, analytical, and QC issues that are encountered with multiplexed immunoassays in an array format. The range of analytical issues encountered included significant differences between duplicate assays as well as different degrees of reproducibility attributable to a particular analyte, plate, or participant. In some cases, different results were obtained as a result of an additional freeze-thaw cycle or erroneous sample dilution. As the authors note, this points out a potential danger of highly multiplexed formats—namely, that an error in the analytical process can propagate across a large number of results. This has always been the case for preanalytical variation; the clinical laboratory cannot overcome mistakes in labeling or sample handling before specimen receipt. However, the experience described by Klee and colleagues does emphasize the extra care that will be required in designing standard operations for clinical microarrays, particularly when significant manual processing is required. The study of Klee and colleagues used an array of arrays approach to analysis: each well in a 96-well microplate contained 6 or 9 test sites. Small arrays are amenable to the quality control and quality assurance methods devised for more conventional tests. As the array size increases, however, the problems of QC increase significantly. For example, consider an array that tests for 1000 different proteins. An initial requirement for successful QC of this microarray is a set of controls with high, medium, and low levels of each of the 1000 proteins to be tested on the array. Many of the currently used control materials have >80 individual analytes, but by and large these are readily obtained in pure form. Formulating controls suitable for a 1000-protein assay would be problematic owing to the difficulty of obtaining proteins in pure form, as well as solubility issues that may be encountered for a high control or combined concentrations of the 1000 different proteins. One possible solution to this problem is suggested by the authors, who noted significant covariation between plate-specific experimental medians of cohort and plasma control data. That is to say, a change in the measured plasma control value was typically accompanied by a corresponding change in the median experimental value for that plate after correcting for known covariates. This suggests that the plasma control data can provide valuable information about the overall behavior of a plate. Given that the authors also observed that most “badly behaved” measurements appeared to be concentrated on a relatively small proportion of the plates, such measures of global plate behavior may provide a useful first step toward designing clinically acceptable microarray quality control. The more significant future challenge will be devising rules for interpreting multiplex QC data and determining the acceptability of the analytical results from an array-based assay. If we consider our hypothetical 1000-protein array with high, medium, and low controls, standard Westgard rules will yield a failure of approximately 8 analytes on every array based solely on the 3σ rule! If we were to extend to a hypothetical array spanning the entire proteome (assume, for the sake of argument, 30 000 relevant analytes), we could repeat arrays at the rate of 1 per second for the life of the known universe (approximately 13.7 billion years) without coming close to finding an array with no analyte violating the 3σ rule (chances of having no 3σ failures for 30 000 different analytes is approximately 6 × 10–36 at one level per analyte). Conversely, a more modest format such as a 15-analyte array would fail inappropriately only approximately 4 times per 100 arrays. In the latter case, the reduction in manual processing makes a strong case for this device, particularly in an era of cost constraints within the clinical lab. As we have argued elsewhere (7), the solution to the QC challenge posed by larger microarrays will require deeper consideration of the clinical questions that are being addressed. If the microarray is merely used as a convenience device, then the limitations of quality control suggest a practical upper limit to the number of analytes that can be combined into a single assay. On the other hand, if patterns of analytes are being combined to give a single diagnostic answer, then different types of multiplex QC algorithms may be appropriate. Another lesson of this work is the importance of design and manufacture for successful clinical translation of microarrays. The authors and their collaborators developed arrays containing a subset of previously validated reagents to measure their particular analytes of interest. In some cases, analytes were reliably measured with low CVs despite the presence of preanalytical and analytical variation. In contrast, other analytes showed low reproducibility and were sensitive to freeze-thaw or dilution. These limitations should not be taken as an indictment of the microarray per se; rather, they illustrate that great care must be taken in the development of high-quality affinity reagents if they are to function as part of a microarray-based diagnostic device. Even antibodies that have been successfully used in uniplex ELISA assays may be inadequate in a microarray format. Of course, in other cases it may be the analyte itself that is problematic. As larger numbers of proteins or other analytes are measured, we must recognize the increasing likelihood that one or more of them may be unstable under normal laboratory conditions (8). Analysis of susceptibility to preanalytical variation will become more—not less—important as multiplexing becomes more common. Thus, it becomes imperative that both assay validation and QC issues be resolved if large–scale or very large–scale microarray analysis is to emerge as a routine method for the clinical laboratory. In the related area of oligonucleotide arrays, where several million test sites can be present in an array on a single chip, these problems have been ameliorated in part by the use of internal controls in the form of a mismatch oligonucleotide for each test site on the array. We agree with Klee and colleagues that “multiplex measurements present difficult challenges that require further analytical and statistical developments.” The scientific community, although convinced by the power and utility of microarray-based analytical methods, continues to be concerned about the quality of results produced in microarray experiments as evidenced by quality initiatives [e.g., the microarray quality control (MAQC) project (9)] and many recent articles and reviews that focus on the quality of microarray data (10)(11)(12)(13)(14)(15). A vital next step in the evolution of large-scale microarrays is the development of the alternative algorithms advocated by Klee and colleagues (6). Until this is achieved, the full potential of large-scale and very large–scale microarray analysis involving thousands and tens of thousands of test sites per array, especially for proteins, will not be fully realized. Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article. Authors’ Disclosures of Potential Conflicts of Interest:Upon manuscript submission, all authors completed the Disclosures of Potential Conflict of Interest form. Potential conflicts of interest: Employment or Leadership: None declared. Consultant or Advisory Role: None declared. Stock Ownership: None declared. Honoraria: None declared. Research Funding: None declared. Expert Testimony: None declared. Other Remuneration: S.R. Master, Invitrogen. Role of Sponsor: The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, or preparation or approval of manuscript.