Real-Time Stream Processing Platform for Massive Logs of Logistics Services
CSTR:
Author:
Affiliation:

Clc Number:

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    With the rapid development of e-commerce platforms, the logistics industry is at a high rate of growth. The access logs of the logistics service platform can reflect user behavior, so it is very important to tap the hidden information to help the logistics service platform optimize the business. At present, higher real-time requirements are imposed on large-scale log data processing. This study comprehensively considers a variety of stream processing frameworks capable of real-time computing, large-scale storage databases, log collection tools, etc. It chooses Flume and Kafka as the log collection tools and message queues and uses Flink and HBase for real-time calculation of streaming data and large-scale data storage. At the same time, the functions including data deduplication, abnormal alarms, fault tolerance strategy, and load scheduling are designed for the platform. Experimental tests have proved that this processing platform can efficiently process log data of the logistics service platform, with innovative ideas and practical value.

    Reference
    Related
    Cited by
Get Citation

梁方玮,薛涛.面向物流服务的海量日志实时流处理平台.计算机系统应用,2021,30(10):68-75

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:January 03,2021
  • Revised:January 29,2021
  • Adopted:
  • Online: October 08,2021
  • Published:
Article QR Code
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063