Special Issue on Enhancement of Reproducibility and Rigor

This special issue of Cytometry focuses on a review of the current landscape regarding the enhancement of reproducibility and rigor in scientific research, specifically in studies where cytometry is utilized. Since the onset of the initiative from NIH toward "Enhancing Reproducibility through Rigor and Transparency" 1, there have been hundreds of publications spanning many scientific disciplines focusing on this issue. A PubMed search using "rigor" and "reproducibility" keywords in 2019 returned almost 500 publications. While initiated by NIH, this quest for better reproducibility in science has impact worldwide. Prior to the NIH directive, the flow cytometry community itself recognized the need for standardization and was already well underway in their attempts toward standardization and the minimization of variability. Publications specific to flow cytometry show peaks in years when specific types of cytometric measurements or techniques were being addressed in the pursuit of more reproducible results. Concern about the variability of results related to flow cytometry had been recognized as a roadblock toward the development of cell-based therapeutics 2. As a vibrant, constantly evolving discipline, this field has often faced challenges in variability and standardization, ultimately producing specific publications centered on these issues. A review of the literature shows peaks in the number of publications related to standardization of these specialized cytometric techniques. Peaks of publications related to standardization toward reproducibility in flow cytometry occurred in 1993 with a focus on cell cycle analysis, in 1998 centered around quantitative flow cytometry and calibration methods, while in 2013 a peak of publications focused on validation of cell-based fluorescence assays, instrument variability, and its impact on immunophenotyping. Most recently, there has been a peak in publications embracing quality assurance for flow cytometry in general 2. For those involved in the design, implementation, analysis and reporting of data generated using cytometric methods, there are multiple facets of the process where a detailed review of these processes can lead to enhancements in the process, proving beneficial toward our contribution to the reproducibility of the research. The review of current processes, with a critical eye toward improvement for better rigor, transparency, and reproducibility, is a responsibility of the technician and the investigator; the reagent, instrument, and software vendors; and, ultimately, in the case of published research, the editors of scientific journals. Since cell sorting is an important utilization of flow cytometry, rigor and reproducibility must be considered in this practice also and is essential in the field of cellular therapeutics. An appreciation of the possible manifestation of change in function of the sorted material, or sorter-induced cellular stress (SICS), needs to be understood and factored into the reproducibility equation. In all phases under the control of practitioners of flow cytometry, we all must be diligent in asking the hard questions. Have we considered and controlled for all of the variables related to this experiment? Are the reagents appropriate and verifiable? Will our instrumentation consistently produce reproducible data which can be calibrated back to a known standard? Can our description of the process we utilized be reproduced with similar results once out of our hands? The answers to these questions should be reviewed throughout the entire scientific process but often are glossed over when core laboratories are utilized due to the dynamics observed in that setting between the process of generating data and the lack of attribution of that data to the core laboratory. The urgency to publish may cause investigators to lose sight of the important factors that are critical to attribution and reproducibility of generated data. In the core laboratory setting, where educating users to perform their own experiments or cell sorting is often an important function, these issues are a regular concern. Due to the often large number of laboratories serviced, as well as the variety of services offered, the core laboratory has the challenge of bringing best practices to the masses, and then the task of monitoring compliance in publications where the core laboratory's contribution may not be acknowledged. Notwithstanding the multitude of opportunities, we have to examine and improve the technical aspects of cytometry for the sake of reproducibility and rigor in scientific research, we must also consider opportunities for improvement in other phases of the process. The interaction of the flow cytometry laboratory with other downstream technologies, in particular where sorted material is used for genomics or metabolomics studies, can constitute a missed opportunity without establishment of robust communication channels between core laboratories of different disciplines within the same institution. We must consider the many aspects of the communication process within our own discipline and across disciplines. While improvements have been made toward the acknowledgment of core laboratories in published research, any omission of core laboratories' role injects potential negative impact on reproducibility and attribution. Related to this lack of acknowledgement are issues when flow cytometric data are not deposited in the appropriate public data repository. Consider the difficulty one encounters if asked to reproduce published flow cytometric data where the core laboratory was not acknowledged, data were not deposited in the Flow Repository with no MiFlowCyt 3 information is available, and flow cytometric methodology was written without input from the core laboratory. This lack of data attribution could effectively be reversed if before publication, journal editors asked, "Were core laboratories utilized in the preparation of the submitted manuscript?" Authors could simply tick a box indicating yes or no. A "yes" answer could conceivably be utilized for generation of an automated acknowledgment of the core laboratory's involvement and could add supporting grant information. A "no" answer could serve to advise the editor to seek additional reviewers for the manuscript with up-to-date knowledge of the technological discipline utilized. Of course for this practice to be effective, core laboratories need to operate according to a recognized set of standards of best practices with a focus on reproducibility and should be recognized for this process by a sanctioning body. We are advancing toward this goal. While ISAC is actively developing a recognition program for cytometric core laboratories that embrace discipline-adopted best practices, the Association of Biomedical Resource Facilities (ABRF) has taken a complementary step, assembling groups of technology experts to be available as part of the review process, representing all the core technological disciplines typically involved in biomedical research, including flow cytometry. We hope that the papers in this Special Issue of Cytometry inspire those involved in the experimental design, execution, analysis, and communication of data to give extra concern toward the reproducibility of their efforts.