###
计算机系统应用:2018,27(10):133-139
本文二维码信息
码上扫一扫!
基于Spark Streaming的实时交通数据处理平台
谭亮, 周静
(四川省交通运输发展战略和规划科学研究院, 成都 611130)
Real-Time Traffic Data Processing System Based on Spark Streaming
TAN Liang, ZHOU Jing
(Sichuan Provincial Institute of Transportation Development Strategy and Planning, Chengdu 611130, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 67次   下载 76
投稿时间:2018-03-09    修订日期:2018-04-03
中文摘要: 交通大数据是解决城市交通问题的最基本条件,是制定宏观城市交通发展战略规划和进行微观道路交通管理与控制的重要保障.针对于智能交通系统中数据产生快、实时性强、数据量大的特点,本文基于Spark Streaming和Apache Kafka的组合构建了一个实时交通数据处理平台,用于处理通过双基基站采集的数据,采用时间窗口机制从持续的Kafka分布式消息队列中获取数据,并按照规则将数据分类处理后保存到数据库.本文对平台的系统架构和内部结构进行了详细的介绍,并通过实验验证了系统的实时处理能力,完全可以在大规模高并发的数据流下进行应用.
Abstract:Traffic big data is the most basic condition for solving urban traffic problems. It is an important guarantee for formulating macro-city traffic development strategy and construction planning. And it is also an important guarantee for carrying out micro-road traffic management and control. In view of the characteristics of intelligent transportation system such as fast data generation, high real-time performance and large amount of data, this paper constructs a real-time traffic data processing platform based on the combination of Spark Streaming and Apache Kafka to process the data collected by dual base stations. Using time window mechanism to get data from Kafka, and process the data to the database according to the rules. In this study, the system architecture and internal structure of the platform are introduced in detail, and the real-time processing capability of the system is verified through experiments, which can be applied under large-scale and high-concurrency data flow.
文章编号:     中图分类号:    文献标志码:
基金项目:
引用文本:
谭亮,周静.基于Spark Streaming的实时交通数据处理平台.计算机系统应用,2018,27(10):133-139
TAN Liang,ZHOU Jing.Real-Time Traffic Data Processing System Based on Spark Streaming.COMPUTER SYSTEMS APPLICATIONS,2018,27(10):133-139

用微信扫一扫

用微信扫一扫