###
DOI:
计算机系统应用英文版:2013,22(12):149-154
本文二维码信息
码上扫一扫!
一种基于分布式平台Hadoop的矩阵相乘算法
(中国电子科技集团公司 第三十二研究所, 上海 200233)
Algorithm of Distributed Matrix Multiplication Based on Hadoop
(The 32nd Research Institute of China Electronics Technology Group Corporation, Shanghai 200233, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 1352次   下载 3461
Received:May 21, 2013    Revised:June 24, 2013
中文摘要: 为了解决单节点上大矩阵相乘运算量过大的问题,提出了一种基于分布式平台Hadoop的矩阵相乘算法. 算法采用了字节文件作为输入以及最优化的分片方式,去掉了不必要的Reduce过程,极大地减少了输入数据量,拥有简洁的算法流程和可拓展性. 实验结果表明该算法很好地解决了矩阵相乘运算量过大的问题,当输入数据量接近集群最佳负载量的情况下取得了很好的加速比.
Abstract:In order to release the burden of large matrix multiplication on a single node, a distributed matrix multiplication algorithm based on Hadoop is proposed. The algorithm uses the input file with binary format, applys optimal split, removes the unnecessary Reduce phase. The algorithm can greatly reduce the amount of input data, has simple algorithm flow and good scalability. Experiment results demonstrate that the algorithm greatly reduces the computation of matrix multiplication and achieves good speedup when the amount of input data is around the optimum loading of the cluster.
文章编号:     中图分类号:    文献标志码:
基金项目:国家“863”计划基金重点项目(2009AA012201);上海市科委科技攻关基金重大项目(08dz501600)
引用文本:
冯健,倪明,赵建波.一种基于分布式平台Hadoop的矩阵相乘算法.计算机系统应用,2013,22(12):149-154
FENG Jian,NI Ming,ZHAO Jian-Bo.Algorithm of Distributed Matrix Multiplication Based on Hadoop.COMPUTER SYSTEMS APPLICATIONS,2013,22(12):149-154