Large-Scale Log Collection and Analysis System Based on Docker
CSTR:
Author:
Affiliation:

Clc Number:

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    The traditional log analysis technology has low efficiency, simple function, poor scalability in practice in processing the large-scale log. To solve these problems, a large-scale log collection and analysis system based on Docker is designed. There are five layers including data collection, data cache, data forwarding, data storage, data retrieval and display in the system. And the system can take in any type of log files from different data sources, provide reliable data transmission through Kafka message queue, utilize Elasticsearch to realize distributed storage and retrieval of data, and analyze log by means of visualization. Meanwhile, the use of docker container technology can realize rapid deployment and version control of the system. The system has the characteristics of real-time, scalability, easy deployment and so on. The experimental results show that the system is feasible and effective with good practical value.

    Reference
    Related
    Cited by
Get Citation

罗东锋,李芳,郝汪洋,吴仲城.基于Docker的大规模日志采集与分析系统.计算机系统应用,2017,26(10):82-88

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:January 13,2017
  • Revised:
  • Adopted:
  • Online: October 31,2017
  • Published:
Article QR Code
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063