Events as a Basis for Workflow Scheduling

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

This paper evaluates using event-based scheduling as a basis for dynamic workflow management. To do this, the WorkflowRunner is introduced as a tool for conducting event driven scheduling in a robust manner. It is evaluated in comparison to Slurm and the WorkflowRunner is found to schedule analysis ~2.5x quicker, in the majority of use cases. An example workflow is also presented, demonstrating how this style of scheduling allows for complete modification of the workflow structure at runtime, something very difficult to achieve in traditional workflow management systems. These developments are expected to be of particular use to dynamic scheduling systems, as well as for distributed, or heterogeneous systems such as those looking to accommodate human-in-the-loop interactions.

Original languageEnglish
Title of host publicationProceedings of WORKS 2022 : 17th Workshop on Workflows in Support of Large-Scale Science, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis
Number of pages8
PublisherIEEE
Publication date2022
Pages52-59
ISBN (Electronic)9781665451918
DOIs
Publication statusPublished - 2022
Event17th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2022 - Dallas, United States
Duration: 13 Nov 202218 Nov 2022

Conference

Conference17th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2022
LandUnited States
ByDallas
Periode13/11/202218/11/2022

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

    Research areas

  • Dynamic, Heterogeneous, MEOW, Workflows

ID: 344725898