Recommendation Algorithm Using Matrix Decomposition and Nearest Neighbor Fusion Based on Spark
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [12]
  • |
  • Related
  • | | |
  • Comments
    Abstract:

    With the current rapid development of mobile Internet, the information overload problem that people face is particularly serious, which makes it a big challenge to do particular users' personalized recommendation in the big data scenario. In order to further improve the timeliness, accuracy of recommendation and ease the problem led by large amount of data, we propose a optimized matrix decomposition recommendation algorithm under the environment of big data in this paper. This algorithm integrates users and the similarity computation of items on the basis of the traditional matrix decomposition algorithm. In the process of training objective function, we enhance the recommendation accuracy by taking in account of users and k nearest neighbors' similarity computation of items. Taking Spark's advantage on memory computing and iterative computing, we design an algorithm using matrix decomposition and nearest neighbor fusion under the Spark framework. Experiments conducted on the classical MovieLens dataset show that our proposed algorithm can deal with data sparseness well, improve recommendation accuracy to some extent, and has a better computational efficiency in the comparison with traditional matrix decomposition recommendation algorithms.

    Reference
    1 Schafer JB, Dan F, Herlocker J, et al. Collaborative filtering recommender systems. The Adaptive Web, Methods and Strategies of Web Personalizationk, 2010: 46-45.
    2 Koren Y. Factorization meets the neighborhood: A multifaceted collaborative filtering model. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM. 2008. 426-434.
    3 Bennett J, Lanning S, Netflix N. The Netflix Prize//Kdd Cup and Workshop in Conjunction with Kdd. 2009.
    4 Koren Y, Bell R,Volinsky C. Matrix factorization techniques for recommender systems. Computer, 2009, 42(8): 30-37.
    5 Vozalis MG, Margaritis KG. Applying SVD on item-based filtering. International Conference on Intelligent Systems Design and Applications. 2005. 464-469.
    6 Kim D, Yum BJ. Collaborative filtering based on iterative principal component analysis. Expert Systems with Applications, 2005, 28(4): 823-830.
    7 Herlocker JL, Konstan JA, Borchers A, et al. An algorithmic framework for performing collaborative filtering. SIGIR'99: Proc. of the International ACM SIGIR Conference on Research and Development in Information Retrieval. August 15-19, 1999. Berkeley, Ca, Usa. 1999. 230-237.
    8 杨阳,向阳,熊磊.基于矩阵分解与用户近邻模型的协同过滤推荐算法.计算机应用,2012,32(2):395-398.
    9 张宇,程久军.基于MapReduce的矩阵分解推荐算法研究.计算机科学,2013,40(1):19-21.
    10 Su X, Khoshgoftaar TM. A survey of collaborative filtering techniques. Advances in Artificial Intelligence, 2009, (12).
    11 Zaharia M, Chowdhury M, Franklin MJ, et al. Spark: Cluster computing with working sets. Usenix Conference on Hot Topics in Cloud Computing. USENIX Association. 2010. 1765-1773.
    12 Zaharia M, Chowdhury M, Das T, et al. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. Usenix Conference on Networked Systems Design and Implementation. 2012. 141-146.
    Related
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

王振军,黄瑞章.基于Spark的矩阵分解与最近邻融合的推荐算法.计算机系统应用,2017,26(4):124-129

Copy
Related Videos

Share
Article Metrics
  • Abstract:1445
  • PDF: 2354
  • HTML: 0
  • Cited by: 0
History
  • Received:August 15,2016
  • Revised:September 27,2016
  • Online: April 11,2017
Article QR Code
You are the first1094993Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063