High Availability Dual Engine Data Warehouse Based on Hive
CSTR:
Author:
Affiliation:

Clc Number:

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Breaking isolated information island, integrating heterogeneous data, gathering and sharing exchanges, conducting in-depth analysis and mining, and providing industry-side decision-making and situation analysis have far-reaching theoretical and applied value. Based on the actual demand of the situational awareness service of the Chinese Academy of Sciences, this study designs and implements a Hive-based Hadoop/Spark dual computing engine big data warehouse supporting OLAP analysis in multiple ways, and carries out an optimization design of usability, load balancing, and resource management, which provides platform support for the subsequent data aggregation and mining, knowledge map construction and discipline situation analysis. Experimental results show that the system is flexible, efficient, available, and scalable, the resource scheduling is scientific, and the load balancing effect is obvious.

    Reference
    Related
    Cited by
Get Citation

李翀,张彤彤,杜伟静,刘学敏.基于Hive的高可用双引擎数据仓库.计算机系统应用,2019,28(9):65-71

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:February 28,2019
  • Revised:March 14,2019
  • Adopted:
  • Online: September 09,2019
  • Published: September 15,2019
Article QR Code
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063