本文已被:浏览 1404次 下载 3360次
Received:January 13, 2013 Revised:March 08, 2013
Received:January 13, 2013 Revised:March 08, 2013
中文摘要: MapReduce是云计算中重要的批数据处理框架, 多任务共享MapReduce机群并满足任务实时性要求是调度算法急需解决的问题. 提出两阶段实时调度算法, 将调度划分为任务间调度和任务内调度. 对于任务间调度, 使用抽样法和经验值法确定子任务执行时间, 利用该参数建立资源分配模型, 动态确定任务优先级进行调度; 对于子任务使用延迟调度策略进行调度, 保证计算的本地性. 实验结果显示, 两阶段实时调度算法相比公平调度算法和FIFO算法, 在保证吞吐量的同时能够满足任务实时性要求.
Abstract:MapReduce is a popular batch data processing framework in cloud computing field. Sharing MapReduce cluster and meeting the deadlines of jobs is a key problem to be solved. This paper proposes a two phase real-time scheduling algorithm which separate scheduling into job scheduling and task scheduling.It uses sampling method to estimate the task excuting time so that the scheduler can make a decision on how many slots should be assigned to the job and how to calculate the job's priority. Using delay-scheduling scheme in task scheduling, the"computing locality"problem can be solved well. Experiments result shows that the scheduling algorithm implemented in this paper satisfies the job's real-time requirement as well as throughput of the cluster.
文章编号: 中图分类号: 文献标志码:
基金项目:江苏省产学研前瞻性联合研究项目(BY2009128)
引用文本:
刘吉,陈香兰,代栋,孙明明,周学海.一种MapReduce实时调度算法设计及实现.计算机系统应用,2013,22(8):113-119
LIU Ji,CHEN Xiang-Lan,DAI Dong,SUN Ming-Ming,ZHOU Xue-Hai.Design and Implementation of a Real-time Scheduling Algorithm for MapReduce.COMPUTER SYSTEMS APPLICATIONS,2013,22(8):113-119
刘吉,陈香兰,代栋,孙明明,周学海.一种MapReduce实时调度算法设计及实现.计算机系统应用,2013,22(8):113-119
LIU Ji,CHEN Xiang-Lan,DAI Dong,SUN Ming-Ming,ZHOU Xue-Hai.Design and Implementation of a Real-time Scheduling Algorithm for MapReduce.COMPUTER SYSTEMS APPLICATIONS,2013,22(8):113-119