Events as a Basis for Workflow Scheduling

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Standard

Events as a Basis for Workflow Scheduling. / Marchant, David.

Proceedings of WORKS 2022: 17th Workshop on Workflows in Support of Large-Scale Science, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 2022. p. 52-59.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Harvard

Marchant, D 2022, Events as a Basis for Workflow Scheduling. in Proceedings of WORKS 2022: 17th Workshop on Workflows in Support of Large-Scale Science, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, pp. 52-59, 17th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2022, Dallas, United States, 13/11/2022. https://doi.org/10.1109/WORKS56498.2022.00012

APA

Marchant, D. (2022). Events as a Basis for Workflow Scheduling. In Proceedings of WORKS 2022: 17th Workshop on Workflows in Support of Large-Scale Science, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 52-59). IEEE. https://doi.org/10.1109/WORKS56498.2022.00012

Vancouver

Marchant D. Events as a Basis for Workflow Scheduling. In Proceedings of WORKS 2022: 17th Workshop on Workflows in Support of Large-Scale Science, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE. 2022. p. 52-59 https://doi.org/10.1109/WORKS56498.2022.00012

Author

Marchant, David. / Events as a Basis for Workflow Scheduling. Proceedings of WORKS 2022: 17th Workshop on Workflows in Support of Large-Scale Science, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 2022. pp. 52-59

Bibtex

@inproceedings{97cbe633f998469487c2191e0db98858,
title = "Events as a Basis for Workflow Scheduling",
abstract = "This paper evaluates using event-based scheduling as a basis for dynamic workflow management. To do this, the WorkflowRunner is introduced as a tool for conducting event driven scheduling in a robust manner. It is evaluated in comparison to Slurm and the WorkflowRunner is found to schedule analysis ~2.5x quicker, in the majority of use cases. An example workflow is also presented, demonstrating how this style of scheduling allows for complete modification of the workflow structure at runtime, something very difficult to achieve in traditional workflow management systems. These developments are expected to be of particular use to dynamic scheduling systems, as well as for distributed, or heterogeneous systems such as those looking to accommodate human-in-the-loop interactions. ",
keywords = "Dynamic, Heterogeneous, MEOW, Workflows",
author = "David Marchant",
note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 17th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2022 ; Conference date: 13-11-2022 Through 18-11-2022",
year = "2022",
doi = "10.1109/WORKS56498.2022.00012",
language = "English",
pages = "52--59",
booktitle = "Proceedings of WORKS 2022",
publisher = "IEEE",

}

RIS

TY - GEN

T1 - Events as a Basis for Workflow Scheduling

AU - Marchant, David

N1 - Publisher Copyright: © 2022 IEEE.

PY - 2022

Y1 - 2022

N2 - This paper evaluates using event-based scheduling as a basis for dynamic workflow management. To do this, the WorkflowRunner is introduced as a tool for conducting event driven scheduling in a robust manner. It is evaluated in comparison to Slurm and the WorkflowRunner is found to schedule analysis ~2.5x quicker, in the majority of use cases. An example workflow is also presented, demonstrating how this style of scheduling allows for complete modification of the workflow structure at runtime, something very difficult to achieve in traditional workflow management systems. These developments are expected to be of particular use to dynamic scheduling systems, as well as for distributed, or heterogeneous systems such as those looking to accommodate human-in-the-loop interactions.

AB - This paper evaluates using event-based scheduling as a basis for dynamic workflow management. To do this, the WorkflowRunner is introduced as a tool for conducting event driven scheduling in a robust manner. It is evaluated in comparison to Slurm and the WorkflowRunner is found to schedule analysis ~2.5x quicker, in the majority of use cases. An example workflow is also presented, demonstrating how this style of scheduling allows for complete modification of the workflow structure at runtime, something very difficult to achieve in traditional workflow management systems. These developments are expected to be of particular use to dynamic scheduling systems, as well as for distributed, or heterogeneous systems such as those looking to accommodate human-in-the-loop interactions.

KW - Dynamic

KW - Heterogeneous

KW - MEOW

KW - Workflows

U2 - 10.1109/WORKS56498.2022.00012

DO - 10.1109/WORKS56498.2022.00012

M3 - Article in proceedings

AN - SCOPUS:85147539427

SP - 52

EP - 59

BT - Proceedings of WORKS 2022

PB - IEEE

T2 - 17th IEEE/ACM Workshop on Workflows in Support of Large-Scale Science, WORKS 2022

Y2 - 13 November 2022 through 18 November 2022

ER -

ID: 344725898