Data-Locality Aware Scientific Workflow Scheduling Methods in HPC Cloud Environments
- Authors
- Choi, Jieun; Adufu, Theodora; Kim, Yoonhee
- Issue Date
- Oct-2017
- Publisher
- SPRINGER/PLENUM PUBLISHERS
- Keywords
- Data-aware scheduling; Data-intensive application; Data-locality; Cloud; Scientific workflow
- Citation
- INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING, v.45, no.5, pp 1128 - 1141
- Pages
- 14
- Journal Title
- INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING
- Volume
- 45
- Number
- 5
- Start Page
- 1128
- End Page
- 1141
- URI
- https://scholarworks.sookmyung.ac.kr/handle/2020.sw.sookmyung/5105
- DOI
- 10.1007/s10766-016-0463-0
- ISSN
- 0885-7458
1573-7640
- Abstract
- Efficient data-aware methods in job scheduling, distributed storage management and data management platforms are necessary for successful execution of data-intensive applications. However, research about methods for data-intensive scientific applications are insufficient in large-scale distributed cloud and cluster computing environments and data-aware methods are becoming more complex. In this paper, we propose a Data-Locality Aware Workflow Scheduling ( D-LAWS) technique and a locality-aware resource management method for data-intensive scientific workflows in HPC cloud environments. D-LAWS applies data-locality and data transfer time based on network bandwidth to scientific workflow task scheduling and balances resource utilization and parallelism of tasks at the node-level. Our method consolidates VMs and consider task parallelism by data flow during the planning of task executions of a data-intensive scientific workflow. We additionally consider more complex workflow models and data locality pertaining to the placement and transfer of data prior to task executions. We implement and validate the methods based on fairness in cloud environments. Experimental results show that, the proposed methods can improve performance and data-locality of data-intensive workflows in cloud environments.
- Files in This Item
-
Go to Link
- Appears in
Collections - 공과대학 > 소프트웨어학부 > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.