Enabling converged computing with the Nextflow ecosystem

Efficiently deploying data-intensive computational workflows across heterogeneous computing platforms poses non-trivial challenges.

Traditional high-performance computing (HPC) architectures provide access to “scarce” highly-optimised resources. On the other hand, the emergence of the cloud brought a model characterised by the “abundance” of virtual infinite, elastic computing capacity and infrastructure.

The demand for the optimal deployment of scalable workflow applications in those scenarios is an open problem that requires the convergence to a model that brings the best of both worlds.

Nextflow was designed to enable the deployment of portable and reproducible data pipelines across a variety of different computing platforms. This presentation explains how the Nextflow ecosystem tackles the problem of the efficient deployment of data-intensive pipelines across heterogeneous systems.