本文已被:浏览 1218次 下载 1881次
Received:September 06, 2016 Revised:October 19, 2016
Received:September 06, 2016 Revised:October 19, 2016
中文摘要: 传统的K-means算法虽然具有很多优点,但聚类准则函数对簇密度不均的数据集分类效果较差.文中在加权标准差准则函数的基础之上,增加了收敛性判定,并在Hadoop平台上提出了一种基于MapReduce编程思想设计与优化的K-means并行算法.与传统的K-means算法相比,设计的并行算法在聚类结果的准确性、加速比、扩展性、收敛性等方面都有显著的提高,降低了因簇密度不均引起误分的概率,提高了算法的聚类精度,并且数据规模越大、节点越多,优化的效果就越明显.
Abstract:Although there are many advantages in traditional K-means algorithm, the clustering criterion function has poor efficiency on classification of the data set with uneven cluster density. On the basis of weighted standard deviation criterion function, this paper proposes a K-means parallel algorithm which is designed and optimized based on MapReduce programming. And it also increases the convergence judgment. Compared with the traditional K-means algorithm, the designed parallel algorithm has a significant improvement in the aspects of accuracy, speedup ratio, scalability and the convergence of clustering results. It also reduces the probability of misclassification caused by the uneven cluster density, and improves the clustering accuracy of the algorithm. What's more, the optimization effect will be more obvious when it deals with lager data size and more nodes.
keywords: K-means cluster density clustering accuracy MapReduce Hadoop
文章编号: 中图分类号: 文献标志码:
基金项目:西安市未央区科技计划(201609);陕西省科技计划(2015KTCXSF-10-11)
引用文本:
刘宝龙,苏金.基于Hadoop平台的K-means聚类算法.计算机系统应用,2017,26(6):182-186
LIU Bao-Long,SU Jin.K-Means Clustering Algorithm Based on Hadoop.COMPUTER SYSTEMS APPLICATIONS,2017,26(6):182-186
刘宝龙,苏金.基于Hadoop平台的K-means聚类算法.计算机系统应用,2017,26(6):182-186
LIU Bao-Long,SU Jin.K-Means Clustering Algorithm Based on Hadoop.COMPUTER SYSTEMS APPLICATIONS,2017,26(6):182-186