###
DOI:
计算机系统应用英文版:2013,22(8):113-119
本文二维码信息
码上扫一扫!
一种MapReduce实时调度算法设计及实现
(1.中国科学技术大学 计算机科学与技术学院, 合肥 230027;2.中国科学技术大学 苏州研究院, 苏州 215123)
Design and Implementation of a Real-time Scheduling Algorithm for MapReduce
(1.College of Computer Science and Technology, University of Science and Technology of China, Hefei 230027, China;2.Suzhou Institute for Advanced Study, University of Scienceand Technology of China, Suzhou 215123, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 1404次   下载 3360
Received:January 13, 2013    Revised:March 08, 2013
中文摘要: MapReduce是云计算中重要的批数据处理框架, 多任务共享MapReduce机群并满足任务实时性要求是调度算法急需解决的问题. 提出两阶段实时调度算法, 将调度划分为任务间调度和任务内调度. 对于任务间调度, 使用抽样法和经验值法确定子任务执行时间, 利用该参数建立资源分配模型, 动态确定任务优先级进行调度; 对于子任务使用延迟调度策略进行调度, 保证计算的本地性. 实验结果显示, 两阶段实时调度算法相比公平调度算法和FIFO算法, 在保证吞吐量的同时能够满足任务实时性要求.
中文关键词: MapRuduce  实时调度  抽样法  延迟调度
Abstract:MapReduce is a popular batch data processing framework in cloud computing field. Sharing MapReduce cluster and meeting the deadlines of jobs is a key problem to be solved. This paper proposes a two phase real-time scheduling algorithm which separate scheduling into job scheduling and task scheduling.It uses sampling method to estimate the task excuting time so that the scheduler can make a decision on how many slots should be assigned to the job and how to calculate the job's priority. Using delay-scheduling scheme in task scheduling, the"computing locality"problem can be solved well. Experiments result shows that the scheduling algorithm implemented in this paper satisfies the job's real-time requirement as well as throughput of the cluster.
文章编号:     中图分类号:    文献标志码:
基金项目:江苏省产学研前瞻性联合研究项目(BY2009128)
引用文本:
刘吉,陈香兰,代栋,孙明明,周学海.一种MapReduce实时调度算法设计及实现.计算机系统应用,2013,22(8):113-119
LIU Ji,CHEN Xiang-Lan,DAI Dong,SUN Ming-Ming,ZHOU Xue-Hai.Design and Implementation of a Real-time Scheduling Algorithm for MapReduce.COMPUTER SYSTEMS APPLICATIONS,2013,22(8):113-119