分布式系统Hadoop 平台的视频转码
作者:
基金项目:

国家杰出青年科学基金(60525110);国家973 计划(2007CB307100,2007CB307103);国家自然科学基金(61072057,60902051);中央高校基本科 研业务费专项资金(BUPT2009RC0505);国家科技重大专项(2011ZX03002-001-01,2011ZX03002-002-01)


Distributed Video Transcoding on Hadoop
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [1]
  • |
  • 相似文献
  • |
  • 引证文献
  • | |
  • 文章评论
    摘要:

    在研究了目前主流的视频转码方案基础上,提出了一种分布式转码系统。系统采用HDFS(HadoopDistributed File System)进行视频存储,利用MapReduce 思想和FFMPEG 进行分布式转码。详细讨论了视频分布式存储时的分段策略,以及分段大小对存取时间的影响。同时,定义了视频存储和转换的元数据格式。提出了基于MapReduce 编程框架的分布式转码方案,即Mapper 端进行转码和Reducer 端进行视频合并。实验数据显示了转码时间随视频分段大小和转码机器数量不同而变化的趋势。结

    Abstract:

    Based on study of current video transcoding solutions, we proposed a distributed transcoding system. Video resources are stored in HDFS(Hadoop Distributed File System) and transcoded by MapReduce program using FFMPEG. In this paper, video segmentation strategy on distributed storage and how they affect accessing time are discussed. We also defined metadata of video formats and transcoding parameters. The distributed transcoding framework is proposed on basis of MapReduce programming model. Segmented source videos are transcoding in map tasks and merged into target video in reduce task. Experimental results show that transcoding time is dependent on segmentation size and trascoding cluster size. Compared with single PC, the proposed distributed video transcoding system implemented on 8 PCs can decrease about 80% of the transcoding time.

    参考文献
    1 Ahmad I, Wei XH, Sun Y, Zhang YQ. Video Transcoding: AnOverview of Various Techniques and Research Issues. IEEETrans. on Multimedia, 2005,7(5). 2 杨戈,廖建新,朱晓民.流媒体分发系统关键技术综述.电子学报,2009,37(1):137-145. 3 Barlas G. A Taxonomy and DLT-based analysis of ClusterbasedVideo Trans/Encoding. 14th Euromicro InternationalConference on Parallel, Distributed, and Network-BasedProcessing, PDP 2006,388-395. 4 Cardellini V, Colajanni M, Lancellotti R, Yu PS. A distributedarchitecture of edge proxy servers for cooperative transcoding.The 3rd IEEE Workshop on Internet Applications,2003,66-70. 5 Guo JN, Bhuyan L. Load Sharing in a Transcoding Cluster.Distributed Computing, 2003,835. 6 Sambe Y, Watanabe S, Yu D, Nakamura T. Distributed videotranscoding and its application to grid delivery. Proc.ITC-CSCC2003, 2003, 921-924. 7 Grio African American Breaking News and Opinion,http://www.thegrio.com/. 8 The Apache Hadoop project. http://Hadoop. apache. org/. 9 Ghemawat S, Gobioff H, Leung ST. The Google File System.19th Symposium on Operating Systems Principles. LakeGeorge, New York, 2003,29,43. 10 Hadoop Makes Sense of Lots of Data. http://www.enterprisestorageforum.com/article.php/3890191/Hadoop-Makes-Sense-of-Lots-of-Data.htm 11 Borthakur D. The Hadoop Distributed File System:Architecture and Design. http://hadoop.apache.org/core/docs/current/hdfs_design.html, 2007 12 Dean J, Ghemawat S. MapReduce: Simplied Data Processingon Large Clusters. Proc. of OSDI’04: 6th Symposiumon Operating System Design and Implemention, SanFrancisco, CA, Dec. 2004. 13 Bellard F, FFMPEG multimedia system. http://FFMPEG.sourceforge.net/index.php 14 Mathes A. Folksonomies-cooperative classification andcommunication through shared metadata. Computer MediatedCommunication, LIS590CMC (Doctoral Seminar).Graduate School of Library and Information Science,University of Illinois Urbana-Champaign, December 2004. 15 Matsuyama K, Kraus M, Kitagawa K, Saito N. A Path-BasedRDF Query Language for CC/PP and UAProf. Proc. of the2nd IEEE Annual Conference on Pervasive Computing andCommunications Workshops (PERCOMW’04), 2004.
    相似文献
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

杨帆,沈奇威.分布式系统Hadoop 平台的视频转码.计算机系统应用,2011,20(11):80-85

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2011-03-20
  • 最后修改日期:2011-04-24
文章二维码
您是第11508404位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号