Between Virtual Machines and Serverless Functions: A Heuristic Approach for Scheduling Scientific Workflows in Compute Clouds

Scientific workflows are abstractions that represent steps of computer simulations through DAGs and are executed by specialized systems. Due to the high computational demand of workflows, they are often executed in high-performance computing environments, e.g., clouds, which offer elasticity and resources on demand. However, new cloud resources, such as serverless functions, are not yet fully exploited by the systems responsible for executing workflows. To fill this gap, this article proposes the constructive heuristic FLEXE, which scales workflow activations in the cloud, integrating the use of virtual machines (VMs) and serverless functions. The approach allows you to optimize the execution time or the financial cost (or even weigh both, according to the defined priority), considering the characteristics and limitations of each resource. Experiments with a real workflow in the bioinformatics area demonstrated gains in performance or cost, depending on the objective configured by the user.