An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows

Shuqing Gu; Likai Yao; Cihan Tunc; Ali Akoglu; Salim Hariri; Elizabeth Ritchie

doi:10.1109/ICCAC.2016.23

An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows

Shuqing Gu
, Likai Yao
, Cihan Tunc
, Ali Akoglu
, Salim Hariri
, Elizabeth Ritchie

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Scopus citations

Abstract

Parameter selection is a critical task in scientific workflows in order to maintain the accuracy of the simulation in an environment where physical conditions change dynamically such as in the case of weather research and forecast (WRF) simulations. Considering the large number of simulation parameters, the size of the configuration search space becomes prohibitive for rapidly evaluating and identifying the parameter configuration that leads to most accurate prediction. We present an autonomic workflow performance manager that can automatically manage model initialization and workflow execution for a given resource allocation. We model the configuration selection of WRF workflow using Apache Storm and automate the process of model initialization, configuration and execution. We reduce the timescale of the configuration search workflow by a factor of 10x by using 20 threads when compared to serial workflow execution as it is typically performed by domain scientists.

Original language	English (US)
Title of host publication	Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016
Subtitle of host publication	Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016
Editors	Indranil Gupta, Yixin Diao
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	111-114
Number of pages	4
ISBN (Electronic)	9781509035366
DOIs	https://doi.org/10.1109/ICCAC.2016.23
State	Published - Dec 5 2016
Event	2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016 - Augsburg, Germany Duration: Sep 12 2016 → Sep 16 2016

Publication series

Name	Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016

Other

Other	2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016
Country/Territory	Germany
City	Augsburg
Period	9/12/16 → 9/16/16

Keywords

Apache Storm
Automated Workflow Management
Model Parameter Sweeping
Weather Research and Forecast

ASJC Scopus subject areas

Artificial Intelligence
Computer Networks and Communications

Access to Document

10.1109/ICCAC.2016.23

Cite this

Gu, S., Yao, L., Tunc, C., Akoglu, A., Hariri, S., & Ritchie, E. (2016). An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows. In I. Gupta, & Y. Diao (Eds.), Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016 (pp. 111-114). Article 7774966 (Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCAC.2016.23

An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows. / Gu, Shuqing; Yao, Likai; Tunc, Cihan et al.
Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016. ed. / Indranil Gupta; Yixin Diao. Institute of Electrical and Electronics Engineers Inc., 2016. p. 111-114 7774966 (Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Gu, S, Yao, L, Tunc, C, Akoglu, A, Hariri, S & Ritchie, E 2016, An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows. in I Gupta & Y Diao (eds), Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016., 7774966, Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016, Institute of Electrical and Electronics Engineers Inc., pp. 111-114, 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016, Augsburg, Germany, 9/12/16. https://doi.org/10.1109/ICCAC.2016.23

Gu S, Yao L, Tunc C, Akoglu A, Hariri S, Ritchie E. An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows. In Gupta I, Diao Y, editors, Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 111-114. 7774966. (Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016). doi: 10.1109/ICCAC.2016.23

Gu, Shuqing ; Yao, Likai ; Tunc, Cihan et al. / An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows. Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016. editor / Indranil Gupta ; Yixin Diao. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 111-114 (Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016).

@inproceedings{c5073d149d1a4ee193e1317eb3240346,

title = "An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows",

abstract = "Parameter selection is a critical task in scientific workflows in order to maintain the accuracy of the simulation in an environment where physical conditions change dynamically such as in the case of weather research and forecast (WRF) simulations. Considering the large number of simulation parameters, the size of the configuration search space becomes prohibitive for rapidly evaluating and identifying the parameter configuration that leads to most accurate prediction. We present an autonomic workflow performance manager that can automatically manage model initialization and workflow execution for a given resource allocation. We model the configuration selection of WRF workflow using Apache Storm and automate the process of model initialization, configuration and execution. We reduce the timescale of the configuration search workflow by a factor of 10x by using 20 threads when compared to serial workflow execution as it is typically performed by domain scientists.",

keywords = "Apache Storm, Automated Workflow Management, Model Parameter Sweeping, Weather Research and Forecast",

author = "Shuqing Gu and Likai Yao and Cihan Tunc and Ali Akoglu and Salim Hariri and Elizabeth Ritchie",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016 ; Conference date: 12-09-2016 Through 16-09-2016",

year = "2016",

month = dec,

day = "5",

doi = "10.1109/ICCAC.2016.23",

language = "English (US)",

series = "Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "111--114",

editor = "Indranil Gupta and Yixin Diao",

booktitle = "Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016",

}

TY - GEN

T1 - An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows

AU - Gu, Shuqing

AU - Yao, Likai

AU - Tunc, Cihan

AU - Akoglu, Ali

AU - Hariri, Salim

AU - Ritchie, Elizabeth

PY - 2016/12/5

Y1 - 2016/12/5

N2 - Parameter selection is a critical task in scientific workflows in order to maintain the accuracy of the simulation in an environment where physical conditions change dynamically such as in the case of weather research and forecast (WRF) simulations. Considering the large number of simulation parameters, the size of the configuration search space becomes prohibitive for rapidly evaluating and identifying the parameter configuration that leads to most accurate prediction. We present an autonomic workflow performance manager that can automatically manage model initialization and workflow execution for a given resource allocation. We model the configuration selection of WRF workflow using Apache Storm and automate the process of model initialization, configuration and execution. We reduce the timescale of the configuration search workflow by a factor of 10x by using 20 threads when compared to serial workflow execution as it is typically performed by domain scientists.

AB - Parameter selection is a critical task in scientific workflows in order to maintain the accuracy of the simulation in an environment where physical conditions change dynamically such as in the case of weather research and forecast (WRF) simulations. Considering the large number of simulation parameters, the size of the configuration search space becomes prohibitive for rapidly evaluating and identifying the parameter configuration that leads to most accurate prediction. We present an autonomic workflow performance manager that can automatically manage model initialization and workflow execution for a given resource allocation. We model the configuration selection of WRF workflow using Apache Storm and automate the process of model initialization, configuration and execution. We reduce the timescale of the configuration search workflow by a factor of 10x by using 20 threads when compared to serial workflow execution as it is typically performed by domain scientists.

KW - Apache Storm

KW - Automated Workflow Management

KW - Model Parameter Sweeping

KW - Weather Research and Forecast

UR - https://www.scopus.com/pages/publications/85010303869

UR - https://www.scopus.com/pages/publications/85010303869#tab=citedBy

U2 - 10.1109/ICCAC.2016.23

DO - 10.1109/ICCAC.2016.23

M3 - Conference contribution

T3 - Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016: Co-located with the 10th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2016

SP - 111

EP - 114

BT - Proceedings - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016

A2 - Gupta, Indranil

A2 - Diao, Yixin

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2016 International Conference on Cloud and Autonomic Computing, ICCAC 2016

Y2 - 12 September 2016 through 16 September 2016

ER -

An Autonomic Workflow Performance Manager for Weather Research and Forecast Workflows

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this