The National Nextflow Tower Service for Australian researchers

In 2022 the Australian Biocommons conducted a national survey of research communities on bioinformatics workflows. The responses indicated Nextflow was the most popular language for developing and reusing workflows. Subsequent consultations revealed a need for a shared platform to launch, manage, and monitor Nextflow pipelines across on-premises, as well as both national and commercial HPC and cloud infrastructures.

In response, the BioCommons started a project in partnership with Seqera to build a national Tower service for the Australian bioinformatics research community. The ambition is to provide a suite of three services: (1) personal workflow spaces; (2) organisational collaborative spaces; and (3) a national bioinformatics analysis service. Each service has distinct use cases and support models and are designed to align with needs of the various communities and organisations the BioCommons supports.

To date, 40 early adopters from 19 different research groups and organisations have been onboarded from the Walter and Eliza Hall Institute of Medical Research, Australian Genome Research Facility, Telethon Kids Institute, University of Melbourne Centre for Cancer Research and Children’s Cancer Institute.

These successes notwithstanding, the diversity of the user base and the complexity of the national compute landscape present various implementation challenges. These include: (1) addressing the disparate data migration requirements between different compute infrastructures and data locations; (2) providing flexible configuration to support easier sharing of workflows across users with different levels of expertise; and (3) the heavy lifting of onboarding groups onto Tower.

Our activities in 2024 will see a focus on scaling up and hardening the service, alongside integration efforts to support interoperability with other bioinformatics services such as our reference genome data portal, the Gen3 human data platform and Galaxy Australia.