Workflow Manager Adoption and Best Practices for Bioinformatics Core Facilities

The increased complexity and scale of bioinformatics data analyses presents significant challenges to bioinformatics core facilities. These include managing high workloads, diverse research projects, trainee education, a range of resource constrained computing environments, and maintenance of reproducible analyses. Workflow management systems, in particular Nextflow and its associated nf-core pipelines, have enabled cores to address these challenges. To assess the current landscape of workflow manager adoption and identify common pain-points, the Genomics Bioinformatics Resource Group (GBIRG) surveyed 29 bioinformatics core directors and scientists in fall of 2024. The majority of participants reported using a workflow manager, with Nextflow being the most commonly adopted framework. Participants commonly leveraged existing pipelines, while also developing custom workflows when required. Common challenges among respondents included computing environment configuration, lack of documentation and training materials, and management of computing resources. Based on collective experience across GBIRG members, we provide recommendations for adopting, executing, and developing Nextflow pipelines, covering key points such as pipeline configuration, parameter customization, and contributing new pipelines and modules to the community. We also highlight emerging resources including the Seqera platform, Co-Scientist, and community-maintained institutional profiles as resources that can reduce setup burden and support both pipeline development and user training.