Challenges and Opportunities in Proteomics Data Analysis

Accurate, consistent, and transparent data processing and analysis are integral and critical parts of proteomics workflows in general and for biomarker discovery in particular. Definition of common standards for data representation and analysis and the creation of data repositories are essential to compare, exchange, and share data within the community. Current issues in data processing, analysis, and validation are discussed together with opportunities for improving the process in the future and for defining alternative workflows. Accurate, consistent, and transparent data processing and analysis are integral and critical parts of proteomics workflows in general and for biomarker discovery in particular. Definition of common standards for data representation and analysis and the creation of data repositories are essential to compare, exchange, and share data within the community. Current issues in data processing, analysis, and validation are discussed together with opportunities for improving the process in the future and for defining alternative workflows. Proteomics has undergone tremendous advances over the past few years, and technologies have noticeably matured. Despite these developments, biomarker discovery remains a very challenging task due to the complexity of the samples (e.g. serum, other bodily fluids, or tissues) and the wide dynamic range of protein concentrations. To overcome these issues, effective sample preparation (to reduce complexity and to enrich for lower abundance components while depleting the most abundant ones), state-of-the-art mass spectrometry instrumentation, and extensive data processing and data analysis are required. Most of the serum biomarker studies performed to date seem to have converged on a set of proteins that are repeatedly identified in many studies and that represent only a small fraction of the entire blood proteome. At the 2005 American Society for Mass Spectrometry Asilomar Conference several speakers stressed the bottlenecks of current proteomics strategies and the imminent need for standards (and base-line criteria) to allow benchmarking the various approaches and to compare results obtained by different laboratories. The need for thorough data generation was emphasized. It requires rigorous analytical chemistry tools, the use of instrumentation that ensures high data quality, and consistent and transparent analysis of the generated data. Furthermore experimental design should be improved to generate statistically meaningful results, namely by avoiding overfitting data or by using distinct training and validation data sets. Therefore, a successful biomarker discovery program relies on the experimental design, its execution using high performance instrumentation, and the processing and analysis of the data using refined tools. Currently data analysis remains a major bottleneck. Historically proteomics analyses have focused on the identification of proteins in the context of a specific experiment typically in a single laboratory. More recently, the need for more global, quantitative, and comparative studies has been recognized, and the value of comparing proteomics data across studies and laboratories has been highlighted especially in biomarker studies affecting different disease sites. However, the meaningful comparison, sharing, and exchange of data or analysis results obtained on different platforms or by different laboratories remain cumbersome mainly due to the lack of standards for data formats, data processing parameters, and data quality assessment. The necessity of an integrated pipeline for processing and analysis of complex proteomics data sets has therefore become critical. Here we briefly describe current art in proteomics data analysis and its integration into a continuous linear pipeline while underscoring current issues and pointing out opportunities for the near future. Processing and analysis of proteomics data is indeed a very complex, multistep process (1Kearney P. Thibault P. Bioinformatics meets proteomics—bridging the gap between mass spectrometry data analysis and cell biology.J. Bioinform. Comput. Biol. 2003; 1: 183-200Crossref PubMed Scopus (57) Google Scholar, 2Listgarten J. Emili A. Statistical and computational methods for comparative proteomic profiling using liquid chromatography-tandem mass spectrometry.Mol. Cell. Proteomics. 2005; 4: 419-434Abstract Full Text Full Text PDF PubMed Scopus (244) Google Scholar, 3Baldwin M.A. Protein identification by mass spectrometry.Mol. Cell. Proteomics. 2004; 3: 1-9Abstract Full Text Full Text PDF PubMed Scopus (172) Google Scholar). The consistent and transparent analysis of LC/MS and LC/MS/MS data requires multiple stages as indicated in Fig. 1. It includes processing of the raw data to extract the relevant signals and information, database searches to assign the spectra to peptide sequences, reassembly in silico of the identified peptides into proteins, validation of the search results at the peptide and protein levels, and annotation and storage of the results in a database. This process remains the main bottleneck for many larger proteomics studies. Ideally modules solving each one of these tasks should be integrated into a linear process like the Trans-Proteome Pipeline, which allows smooth processing of the data through the different stages (4Keller A. Eng J. Zhang N. Aebersold R. A uniform proteomics MS/MS analysis platform utilizing open XML file formats.Mol. Syst. Biol. 2005; 1 (2005. 0017)Crossref PubMed Scopus (599) Google Scholar). Several factors can lead to significant improvements in the data analysis, including the full leverage of latest instrument capabilities to generate high quality data, the definition of platform-independent data format, and the use of standardized, transparent, and generally available data analysis protocols and tools that will produce consistent and comparable results. Despite the complexity of the process, it is likely to improve rapidly as the analysis pipeline evolves and data quality increases. An important, frequently underestimated element of an integrated data analysis system is the data acquisition and signal preprocessing. A related issue is the format used to capture and store the data. Often instruments are operated as a black box and are not always used to the maximum of their performance while data preprocessing is often performed in default mode. For instance, data quality increases significantly (at the expenses of data volume) by acquiring data in profile mode and by subsequently (postacquisition) using more elaborate algorithms to determine signal and noise and derive more accurate measurements. Peak detection (peak picking) is a key element, often neglected, in the data analysis. It is usually part of the instrument software, and the users have limited control over it. This step is often performed automatically during the data acquisition, and the critical parameters are not always explicitly documented. However, high quality data combined with effective preprocessing tools (i.e. algorithms for noise reduction, peak detection, and monoisotopic peak determination) are the basis of a reliable data analysis. The maxim “garbage in-garbage out” retains its full meaning is this context. High quality raw data (e.g. profile mode) together with refined peak detection algorithms allow reliable determination of charge state, monoisotopic m/z values, and signal intensities of peptide ions. In practice, much better results are obtained by collecting the data first and then reprocessing them off line to fully take advantage of the capabilities of modern instrumentation, which can drastically improve both identification and quantification. It includes higher sensitivity, high resolution, and high mass accuracy, which should be fully retained and exploited during the downstream data analysis. High quality data combined with advanced data processing tools are critical for a deeper insight into proteomics samples in general and in serum or plasma samples more specifically. A large variety of instrument platforms (ion traps, quadrupole/time-of-flight, ion cyclotron resonance, time-of-flight/time-of-flight, etc.) from various manufacturers are available for proteomics studies. Each instrument type will generate spectra with its own characteristics (signal-to-noise ratio, resolution, accuracy, etc.) usually in a proprietary data format. Data processing algorithms are not fully documented and usually are restricted to one instrument platform, thus limiting portability to other data processing tools and comparison of results. The definition of a generic mass spectrometric data format such as mzXML 1The abbreviations used are: XML, extensible markup language; MRM, multiple reaction monitoring. (5Pedrioli P.G.A. Eng J.K. Hubley R. Vogelzang M. Deutsch E.W. Raught B. Pratt B. Nilsson E. Angeletti R.A. Apweiler R. Cheung K. Costello C.E. Hermjakob H. Huang S. Julian Jr, R.K. Kapp E. McComb M. Oliver S.G. Omenn G. Paton N.W. Simpson R. Smith R. Taylor C.F. Zhu W. Aebersold R.A. Common open representation of mass spectrometry data and its application to proteomics research.Nat. Biotechnol. 2004; 22: 1459-1466Crossref PubMed Scopus (655) Google Scholar) and the Human Proteome Organization’s Proteomics Standards Initiative (6Orchard S. Kersey P. Hermjakob H. Apweiler R. Proteomics Standards Initiative meeting: towards common standards for exchanging proteomics data.Comp. Funct. Genomics. 2003; 4: 16-19Crossref PubMed Scopus (26) Google Scholar) have been first steps to overcome this problem. The use of a standardized file format allows analyzing data within a pipeline that is independent of the instrument platform. Although the conversion into the mzXML format requires additional computing resources and may increase the file size, a generic format broadly accepted by the community, including the manufacturers, will foster sharing and exchanging data in the future. In this context, the concrete plan to merge the mzXML and mzDATA formats into a single unified file format is encouraging. The second main step consists of the assignment of MS/MS spectra to peptide sequences by submitting the MS/MS spectra to a database search using one of several engines available (e.g. Sequest, Mascot, Comet, X!tandem, etc.). Most approaches are matching and scoring large sets of experimental spectra with predicted masses of fragment ions of peptide sequences derived from a protein database. Results are scored according to a scheme specific to each search engine that also depends on the database used for the search. Usually tools are linked to one specific platform or were optimized for one instrument type. The various search engines do not yield identical results as they are based on different algorithms and scoring functions, making comparison and integration of results from different studies or experiments tedious. To compare results (and to some extent search engine performances) searches have to be performed under well defined and comparable parameters. Recently published guidelines for database searching have addressed this issue (7Carr S. Aebersold R. Baldwin M. Burlingame A. Clauser K. Nesvizhskii A. The need for guidelines in publication of peptide and protein identification data: Working Group on Publication Guidelines for Peptide and Protein Identification Data.Mol. Cell. Proteomics. 2004; 3: 531-533Abstract Full Text Full Text PDF PubMed Scopus (414) Google Scholar, 8Bradshaw R.A. Burlingame A.L. Carr S. Aebersold R. Reporting protein identification data: the next generation of guidelines.Mol. Cell. Proteomics. 2006; 5: 787-788Abstract Full Text Full Text PDF PubMed Scopus (204) Google Scholar). It is critical at this point that the parameters used for the search are fully tracked and documented. Peptide identification via database searches is very computationally intensive and time-demanding. High quality data allow more effective searches due to tighter constrains, i.e. tolerance on precursor ion mass and charge state assignment, which will drastically reduce the search time in case of an indexed database. In addition, accurate mass measurements of fragment ions further simplify the database searches and add confidence to the results. Once the initial output of the database search engine has been obtained, it is essential that the reliability of the assignments of spectra to peptide sequences is statistically validated. Such analyses generate reliable estimates of the false positive and false negative error rates, values that are critical to meaningfully compare results from multiple experiments or platforms. The PeptideProphet algorithm (9Keller A. Nesvizhskii A. Kolker E. Aebersold R. Empirical statistical model to estimate the accuracy of peptide identifications made by MS/MS and database search.Anal. Chem. 2002; 74: 5383-5392Crossref PubMed Scopus (3912) Google Scholar) has be designed to achieve this goal. The error rates in a data set can also be estimated by performing a search using a “reversed database” (i.e. a database in which the sequences were scrambled to produce only false positive identifications and thus ascertain the false positive error rate (10Elias J.E. Hass W. Faherty B.L. Gygi S.P. Comparative evaluation of mass spectrometry platforms used in large scale proteomics investigations.Nat. Methods. 2005; 2: 667-675Crossref PubMed Scopus (607) Google Scholar)). In contrast to more specialized tools, reversed database search results do not estimate the false negative error rate of a dataset. An alternate strategy consists of storing MS/MS data in a library at an earlier stage in the identification process. Comparison of MS/MS data occurs by comparing experimental spectra with those previously measured and stored in a database using a spectra-matching algorithm. Such an approach was proven to be very effective for several decades in the small molecule area. Stein et al. (11Stein S. Kilpatrick L. Neta P. Roth J. Building and using reference libraries of peptide mass spectra.Proceedings of the 53rd ASMS Conference on Mass Spectrometry, San Antonio, TX, June 5–9, 2005. American Society for Mass Spectrometry, Santa Fe, NM2005Google Scholar) have explored that route by building a library of consensus peptide spectra (i.e. a set of consistent spectra derived from multiple experimental data sets measured on quadrupole ion traps and quadrupole time-of-flight instruments). An extension of the library together with the ability to produce easily comparable results requires some normalization of the parameters used for the data generation Despite this identification of peptides is much and database search one of such a on current instruments with data acquisition to and ions based on their MS/MS that a single such as the mass of the A matching approach is likely to be of the search engine or the protein database The of such an approach are the and quality of the spectra in the the of confidence in the peptide assignments the need for a and the performance of the matching algorithm in false positive and false negative Such libraries will also be resources for multiple reaction approaches as they will simplify the of the i.e. precursor ion The of identified peptides with their precursor proteins is a very critical and step in proteomics strategies as many peptides are common to several proteins, thus to protein it critical to have an that is to the of the protein and a to it. to peptides identified by MS/MS to accurate for the proteins A. A. Kolker E. Aebersold R. A statistical model for proteins by mass Chem. 2003; PubMed Scopus Google Scholar). peptides that have a reliable and from proteins the of proteins that the peptides Aebersold R. of proteomics data: the protein Cell. Proteomics. 2005; 4: Full Text Full Text PDF PubMed Scopus Google Scholar). proteins with multiple peptide have a much confidence in their assignment proteins identified by one single In is a of the false positive error rate at the protein with the peptide This the of this step and the need for a that is to peptide and protein Aebersold R. of proteomics data: the protein Cell. Proteomics. 2005; 4: Full Text Full Text PDF PubMed Scopus Google Scholar). is a further critical step in biomarker studies the is on peptides that in between sets of peptides that are much of peptides across multiple data sets is a very task that has not been fully the to identification and and independent main approaches have been The first is based on and requires of the peptides from the various samples sets with different that have different The are then together and in one single LC/MS/MS The of a specific is then from the signal of the signal in the full the in is identified by database searching of the MS/MS The second which is more relevant to larger biomarker studies (i.e. analysis of a larger sets of samples from and disease each sample and then the multiple LC/MS data acquisition under and in an is essential for this The processing of the multiple data sets several issues, including control of and over of for in and normalization of ion to for in sample or instrument or of these are to In a of LC/MS are more peak are and data are The main step consists in matching the ions (i.e. within specific m/z and time across It is a critical task as the high of within a results in that downstream analysis. high quality data (i.e. high mass accuracy and are critical to this process. It is typically by a high of which are in only one or a few are the analysis can be performed in a a of tools have been to such analyses Zhang H. Aebersold R. A for the generation and comparison of peptide from sets of data by liquid spectrometry.Mol. Cell. Proteomics. 2005; 4: Full Text Full Text PDF PubMed Scopus Google Scholar, J. P. M. B. A. R. P. Zhang H. J. A. W. Proteomics an extensible open system for and proteomic data and high 2006; 5: PubMed Scopus Google Scholar). have the of standards to and exchange data and results. proteomics experiments are out in by one single often in an thus making sharing and comparison of results not The lack of common standards and protocols has lead to this and often in of Results were usually as a set of identified proteins (i.e. of peptides identified and with data. the large of such data sets has made publication of results using very and exchange of data and results requires the definition of formats for the data at raw mass spectrometric data, data, and search as well as a better definition of the parameters used for the data processing or the database In this a range of data and results generated by a variety of tools can be into including results of database search and statistical validation tools (e.g. PeptideProphet and In the parameters used for the analysis are essential and have to be data, including annotation and the to be in the database as Data and results repositories such as for Deutsch E.W. Nesvizhskii P. Eng J.K. A. R. E. S. N. E. L. H. B. B. W. L. Zhang H. Aebersold R. with the of peptide sequences obtained by mass Biol. 2005; PubMed Google Proteome R. system for and storing protein identification Proteome 2004; 3: PubMed Scopus Google or Proteomics Bioinformatics P. L. Taylor C.F. W. Hermjakob H. Apweiler R. a of protein and peptide identifications for the proteomics 2005; 5: Google Scholar) exchange of results and limited to peptide sequences and proteins with a of parameters (i.e. charge state, and signal these are rapidly to to the proteins, and data. can the of parameters as as normalization methods are For instance, have limited value they are into values that can be across the entire database. standards that the can to and integrated platforms at platforms that can the different are of the main in results from multiple platforms and laboratories. Despite major advances the and sharing of data issues the protein annotation and remain to be Current on peptide have in limited of identified peptides and proteins major Furthermore these peptides and proteins are identified with high This has the of strategies to overcome this More approaches advantage of the tools are used to generate and then peptide ions that are part of such a at in the of approaches and allows peptides with much data analysis were designed to automatically peptides and proteins in larger data More recently, the in protein has the of analytical strategies that also on the detection and identification of (i.e. of the and its on the Currently most approaches on database searches that in the search More refined approaches based on or other experimental and are sample preparation to the peptides of is usually a critical step in It will be critical to the current proteomics data repositories and to also high confidence on To overcome some of the of current proteomics strategies in to the dynamic range of peptides and the of MS/MS spectra that the ability to alternative mass approaches are acquisition is limited by the detection of a signal in full mode to a ion In strategies by multiple reaction and a ion to the of a peptide with much the precursor ion is not in the full mass on a quadrupole instrument ion the data are by both mass to m/z values to the ion and one specific fragment ion of the peptide of The mass drastically increases the while the of this experiment for an It thus allows detection and of abundance in complex experiments from approaches in that they are (i.e. for or and are experiments or of through matching to (e.g. time or or predicted MS/MS ion was to be a approach to with high and peptides in complex such as a of plasma samples mass spectrometric for major plasma Cell. Proteomics. 2006; 5: Full Text Full Text PDF PubMed Scopus Google Scholar) or a specific B. J. Aebersold R. strategy for and of in of the ASMS Conference of Mass Spectrometry, American Society for Mass Spectrometry, Santa Fe, Scholar). The advances in the of instrument control and data acquisition the to larger of peptides in one single LC/MS R. system for and storing protein identification Proteome 2004; 3: PubMed Scopus Google Scholar). This analysis of at using a approach as in Fig. The strategy high mass spectrometric measurements in the mode and the proteomics that is available in (e.g. data in are used to peptides (i.e. sequences that are to one single and their are used to This large to and peptides (and thus the as in Fig. In such one by a of proteins of the peptides are and the fragment ions are predicted from a to the of the also be that will then be used to generate a of It is thus in one single LC/MS (and thus with to and these discovery well as many other proteomics are often large experiments large data and results be obtained from of several laboratories. It is essential that data exchange and sharing a transparent process. of data through wide use of common formats and use of transparent tools for data processing and analysis with well defined parameters is