融合多种时空自注意力机制的Transformer交通流预测模型
作者:
基金项目:

福建省科技厅对外合作项目(2020I0014)


Transformer Traffic Flow Prediction Model Integrating Multiple Spatiotemporal Self-attention Mechanisms
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [29]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    交通流预测是智能交通系统中实现城市交通优化的一种重要方法, 准确的交通流量预测对交通管理和诱导具有重要意义. 然而, 因交通流本身存在高度时空依赖性而表现出复杂的非线性特征, 现有的方法主要考虑路网中节点的局部时空特征, 忽略了路网中所有节点的长期时空特征. 为了充分挖掘交通流数据复杂的时空依赖, 提出一种融合多种时空自注意力机制的Transformer交通流预测模型(MSTTF). 该模型在嵌入层通过位置编码嵌入时间和空间信息, 并在注意力机制层融合邻接空间自注意力机制, 相似空间自注意力机制, 时间自注意力机制, 时间-空间自注意力机制等多种自注意力机制挖掘数据中潜在的时空依赖关系, 最后在输出层进行预测. 结果表明, MSTTF模型与传统时空Transformer相比, MAE平均降低了10.36%. 特别地, 相比于目前最先进的PDFormer模型, MAE平均降低了1.24%, 能取得更好的预测效果.

    Abstract:

    Traffic flow prediction is an important method for achieving urban traffic optimization in intelligent transportation systems. Accurate traffic flow prediction holds significant importance for traffic management and guidance. However, due to the high spatiotemporal dependence, the traffic flow exhibits complex nonlinear characteristics. Existing methods mainly consider the local spatiotemporal features of nodes in the road network, overlooking the long-term spatiotemporal characteristics of all nodes in the network. To fully explore the complex spatiotemporal dependencies in traffic flow data, this study proposes a Transformer-based traffic flow prediction model called multi-spatiotemporal self-attention Transformer (MSTTF). This model embeds temporal and spatial information through position encoding in the embedding layer and integrates various self-attention mechanisms, including adjacent spatial self-attention, similar spatial self-attention, temporal self-attention, and spatiotemporal self-attention, to uncover potential spatiotemporal dependencies in the data. The predictions are made in the output layer. The results demonstrate that the MSTTF model achieves an average reduction of 10.36% in MAE compared to the traditional spatiotemporal Transformer model. Particularly, when compared to the state-of-the-art PDFormer model, the MSTTF model achieves an average MAE reduction of 1.24%, indicating superior predictive performance.

    参考文献
    [1] Zaki JF, Ali-Eldin A, Hussein SE, et al. Traffic congestion prediction based on hidden Markov models and contrast measure. Ain Shams Engineering Journal, 2020, 11(3): 535–551.
    [2] Ye YN, Chen L, Xue F. Passenger flow prediction in bus transportation system using ARIMA models with big data. Proceedings of the 2019 International Conference on Cyber-enabled Distributed Computing and Knowledge Discovery (CyberC). Guilin: IEEE, 2019. 436–443.
    [3] Razali NAM, Shamsaimon N, Ishak KK, et al. Gap, techniques and evaluation: Traffic flow prediction using machine learning and deep learning. Journal of Big Data, 2021, 8(1): 152.
    [4] Hou Y, Deng ZY, Cui HK. Short-term traffic flow prediction with weather conditions: Based on deep learning algorithms and data fusion. Complexity, 2021, 2021: 6662959.
    [5] Yang LJ, Yang Q, Li YH, et al. K-nearest neighbor model based short-term traffic flow prediction method. Proceedings of the 18th International Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES). Wuhan: IEEE, 2019. 27–30.
    [6] Tang JJ, Chen XQ, Hu Z, et al. Traffic flow prediction based on combination of support vector machine and data denoising schemes. Physica A: Statistical Mechanics and its Applications, 2019, 534: 120642.
    [7] Zhang WB, Yu YH, Qi Y, et al. Short-term traffic flow prediction based on spatio-temporal analysis and CNN deep learning. Transportmetrica A: Transport Science, 2019, 15(2): 1688–1711.
    [8] Chen XY, Xie XS, Teng D. Short-term traffic flow prediction based on ConvLSTM model. Proceedings of the 5th IEEE Information Technology and Mechatronics Engineering Conference (ITOEC). Chongqing: IEEE, 2020. 846–850.
    [9] Zhao LN, Wen XY, Wang YP, et al. A novel hybrid model of ARIMA-MCC and CKDE-GARCH for urban short-term traffic flow prediction. IET Intelligent Transport Systems, 2022, 16(2): 206–217.
    [10] Li ZH, Chen CL, Min Y, et al. Dynamic hidden Markov model for metropolitan traffic flow prediction. Proceedings of the 92nd IEEE Vehicular Technology Conference (VTC2020-Fall). Victoria: IEEE, 2020. 1–5.
    [11] Raskar C, Nema S. Metaheuristic enabled modified hidden Markov model for traffic flow prediction. Computer Networks, 2022, 206: 108780.
    [12] Sun ZY, Hu YJ, Li W, et al. Prediction model for short-term traffic flow based on a K-means-gated recurrent unit combination. IET Intelligent Transport Systems, 2022, 16(5): 675–690.
    [13] Xing YM, Ban XJ, Liu X, et al. Large-scale traffic congestion prediction based on the symmetric extreme learning machine cluster fast learning method. Symmetry, 2019, 11(6): 730.
    [14] 丁男, 高壮林, 许力, 等. 基于数据优先级和交通流密度的异构车联网数据链路层链路调度算法. 计算机学报, 2020, 43(3): 526–536.
    [15] 卢海鹏, 韩莹, 张凯, 等. 基于VMD-BiLSTM-BLS模型的短时交通流预测. 计算机系统应用, 2022, 31(5): 238–245.
    [16] 张维, 袁绍欣, 陶建军, 等. 基于多元因素的Bi-LSTM高速公路交通流预测. 计算机系统应用, 2021, 30(6): 184–190.
    [17] Li ZJ, Li CH, Cui X, et al. Short-term traffic flow prediction based on recurrent neural network. Proceedings of the 2021 International Conference on Computer Communication and Artificial Intelligence (CCAI). Guangzhou: IEEE, 2021. 81–85.
    [18] Li MZ, Zhu ZX. Spatial-temporal fusion graph neural networks for traffic flow forecasting. Proceedings of the 35th AAAI Conference on Artificial Intelligence. AAAI, 2021. 4189–4196.
    [19] Chen C, Xu YB, Zhao JX, et al. Combining random forest and graph wavenet for spatial-temporal data prediction. Intelligent and Converged Networks, 2022, 3(4): 364–377.
    [20] 王雨松, 吴向东, 尤晨欣, 等. 基于DWT-GCN的短时交通流预测. 计算机系统应用, 2022, 31(9): 306–312.
    [21] Song C, Lin YF, Guo SN, et al. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York: AAAI, 2020. 914–921.
    [22] Guo K, Hu YL, Sun YF, et al. Hierarchical graph convolution network for traffic forecasting. Proceedings of the 35th AAAI Conference on Artificial Intelligence. AAAI, 2021. 151–159.
    [23] Vijayalakshmi B, Ramar K, Jhanjhi NZ, et al. An attention-based deep learning model for traffic flow prediction using spatiotemporal features towards sustainable smart city. International Journal of Communication Systems, 2021, 34(3): e4609.
    [24] Reza S, Ferreira MC, Machado JJM, et al. A multi-head attention-based Transformer model for traffic flow forecasting with a comparative analysis to recurrent neural networks. Expert Systems with Applications, 2022, 202: 117275.
    [25] Yan HY, Ma XL, Pu ZY. Learning dynamic and hierarchical traffic spatiotemporal features with Transformer. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 22386–22399.
    [26] Wen YJ, Xu P, Li ZH, et al. RPConvformer: A novel transformer-based deep neural networks for traffic flow prediction. Expert Systems with Applications, 2023, 218: 119587.
    [27] Xu MX, Dai WR, Liu CM, et al. Spatial-temporal Transformer networks for traffic flow forecasting. arXiv:2001.02908, 2020.
    [28] Jiang JW, Han CK, Zhao WX, et al. PDFormer: Propagation delay-aware dynamic long-range Transformer for traffic flow prediction. Proceedings of the 37th AAAI Conference on Artificial Intelligence. Washington: AAAI, 2023. 4365–4373.
    [29] Saha S, Haque A, Sidebottom G. Deep sequence modeling for anomalous ISP traffic prediction. Proceedings of the 2022 IEEE International Conference on Communications. Seoul: IEEE, 2022. 5439–5444.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

曹威,王兴,邹复民,金彪,王小军.融合多种时空自注意力机制的Transformer交通流预测模型.计算机系统应用,2024,33(4):82-92

复制
分享
文章指标
  • 点击次数:806
  • 下载次数: 2884
  • HTML阅读次数: 1191
  • 引用次数: 0
历史
  • 收稿日期:2023-10-08
  • 最后修改日期:2023-11-09
  • 在线发布日期: 2024-01-30
文章二维码
您是第11246044位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号