Improvement of ETL System in NC Machine Sensor Data Analysis
CSTR:
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [13]
  • |
  • Related [20]
  • |
  • Cited by [0]
  • | |
  • Comments
    Abstract:

    ETL(Extract-Transform-Load) is an indispensable component of the NC machine sensor data analysis. In view of the increasing amount of data in recent years, and the rising requirements for real-time data processing, this paper improves the traditional change data capture methods, and designs real-time data capture solutions based on the characteristics of those data. What's more, it details the entire ETL distributed architecture design. Finally, experiments show that the ETL system on NC machine sensor data real-time processing has higher efficiency.

    Reference
    [1] 何刚. 基于Hadoop平台的分布式ETL研究与实现[硕士学位论文]. 上海:东华大学, 2014.
    [2] 宋杰, 郝文宁, 陈刚, 等. 基于MapReduce的分布式ETL体系结构研究. 计算机科学, 2013, 40(6):152-154.
    [3] 张亮. 分布式数据仓库中ETL技术的研究[硕士学位论文]. 沈阳:沈阳航空工业学院, 2009.
    [4] 刘胜. 基于增量ETL的分布式数据交换平台的研究与实现[硕士学位论文]. 北京:国防科学技术大学, 2011.
    [5] 林建昌. 电力行业分布式ETL数据集成系统研究与实现[硕士学位论文]. 成都:电子科技大学, 2015.
    [6] 马瑞新, 许力. 基于SOA的实时ETL的研究与实现. 计算机工程与科学, 2007, 29(8):115-117.
    [7] 段成, 王增平, 吴克河. 一种轻量级电网实时数据ETL系统的设计与实现. 电力系统保护与控制, 2010, 38(18):174-177, 182.[DOI:10.7667/j.issn.1674-3415.2010.18.034]
    [8] Kumar Hota CPP, Ramu Y, Subba Rao DBV. A relative study on traditional ETL and ETL with apache hadoop. NCRTIT2K16. Andhra Pradesh, India. 2016. 74-78.
    [9] Awad MMI, Abdullah MS. A framework for interoperable distributed ETL components based on SOA. Proc. of the 2nd International Conference on Software Technology and Engineering. San Juan, PR, USA. 2010. V1-67-V1-70.
    [10] Kakish K, Kraft TA. ETL evolution for real-time data warehousing. Proc. of the Conference on Information Systems Applied Research. New Orleans Louisiana. 2012.
    [11] Liu XF, Thomsen C, Pedersen TB. ETLMR:A highly scalable dimensional ETL framework based on mapReduce. Proc. 2011 International Conference on Data Warehousing and Knowledge Discovery. Berlin Heidelberg. 2011. 196-3111.
    [12] 章水鑫, 徐宏炳, 于立. 增量式ETL工具的研究与实现. 现代计算机, 2005, (3):6-10.
    [13] 廖飒. 数据仓库增量ETL的设计及优化. 贵阳学院学报(自然科学版), 2008, 3(3):5-9.
    Comments
    Comments
    分享到微博
    Submit
Get Citation

刘峰,蔡明高,于波,于碧辉.数控机床传感器数据分析中ETL系统改进.计算机系统应用,2017,26(9):93-97

Copy
Share
Article Metrics
  • Abstract:1587
  • PDF: 1947
  • HTML: 0
  • Cited by: 0
History
  • Received:December 27,2016
  • Online: October 31,2017
Article QR Code
You are the first991237Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063