基于Q学习的蚁群优化水声网络协议
作者:
基金项目:

上海市教委科技创新项目(2101070010E00121)


Ant Colony Optimization Based on Q-learning for Underwater Acoustic Network Protocol
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [18]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    针对水声通信中数据传输延时高且动态适应性弱的问题, 提出了一种基于Q学习优化的蚁群智能水声网络路由协议(Q-learning ant colony optimization, QACO). 协议包括路由行为和智能决策部分, 在路发现和维护阶段, 依靠网络智能蚂蚁进行网络拓扑环境的构建和节点之间的信息交换以及网络的维护. 在Q学习阶段, 通过定量化节点能量和深度以及网络传输延时学习特征作为折扣因子和学习率, 以延长网络的生命周期, 降低系统能耗和延时. 最后通过水声网络环境进行仿真, 实验结果表明QACO在能耗、延迟和网络生命周期方面都优于基于Q学习辅助的蚁群算法(Q-learning aided ant colony routing protocol, QLACO)和基于 Q-learning 的节能和生命周期感知路由协议(Q-learning-based energy-efficient and lifetime-aware routing protocol, QELAR)和基于深度路由协议 (depth-based routing, DBR)算法.

    Abstract:

    To solve the problems such as high data transmission delay and weak dynamic adaptability of underwater acoustic communication, this study proposes an intelligent underwater acoustic network routing protocol based on Q-learning ant colony optimization (QACO). The protocol includes routing behavior and intelligent decision. In the route discovery and maintenance phase, the construction of the network topology environment and information exchange among nodes as well as the network maintenance are carried out by intelligent NetAnts. In the Q-learning phase, the node energy and depth and network transmission delay learning characteristics are quantified as discount factors and learning rates to extend the network lifecycle and reduce system energy consumption and delay. Finally, simulations are carried out through the underwater acoustic network environment, and the experimental results show that QACO outperforms the Q-learning aided ant colony routing protocol (QLACO), Q-learning-based energy-efficient and lifetime-aware routing protocol (QELAR), and depth-based routing (DBR) algorithm in terms of energy consumption, delay, and network lifecycle.

    参考文献
    [1] Zhang RX, Ma XL, Wang DQ, et al. Adaptive coding and bit-power loading algorithms for underwater acoustic transmissions. IEEE Transactions on Wireless Communications, 2021, 20(9): 5798–5811. [doi: 10.1109/TWC.2021.3070363
    [2] Doosti-Aref A, Ebrahimzadeh A. Adaptive relay selection and power allocation for OFDM cooperative underwater acoustic systems. IEEE Transactions on Mobile Computing, 2018, 17(1): 1–15
    [3] Chen XK, Yang HJ, Wang QM, et al. Network coding-based relay selection and power distribution for underwater cooperative communication networks. Proceedings of the 2020 International Conference on Wireless Communications and Signal Processing. Nanjing: IEEE, 2020. 596–601.
    [4] Xie P, Cui HJ, Lao L. VBF: Vector-based forwarding protocol for underwater sensor networks. Proceedings of the 5th International Conference on Research in Networking. Coimbra: Springer, 2006. 1216–1221.
    [5] Hwang D, Kim D. DFR: Directional flooding-based routing protocol for underwater sensor networks. Proceedings of the OCEANS 2008. Quebec City: IEEE, 2008. 1–7.
    [6] Yan H, Shi ZJ, Cui JH. DBR: Depth-based routing for underwater sensor networks. Proceedings of the 7th International Conference on Research in Networking. Singapore: Springer, 2008. 72–86.
    [7] Yildiz HU, Gungor VC, Tavli B. Packet size optimization for lifetime maximization in underwater acoustic sensor networks. IEEE Transactions on Industrial Informatics, 2019, 15(2): 719–729. [doi: 10.1109/TII.2018.2841830
    [8] Nazareth P, Chandavarkar BR. Link and void aware routing protocol for underwater acoustic sensor networks. Proceedings of the 12th International Conference on Computing Communication and Networking Technologies. Kharagpur: IEEE, 2021. 1–6.
    [9] Shi YM, Rong ZH. Analysis of Q-learning like algorithms through evolutionary game dynamics. IEEE Transactions on Circuits and Systems II: Express Briefs, 2022, 69(5): 2463–2467. [doi: 10.1109/TCSII.2022.3161655
    [10] Dorigo M, Maniezzo V, Colorni A. Ant system optimization by a colony of cooperating agents.IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics). 1996, 26(1): 29–41.
    [11] Lyu L, Shen Y, Zhang SC. The advance of reinforcement learning and deep reinforcement learning. Proceedings of the 2022 IEEE International Conference on Electrical Engineering, Big Data and Algorithms. Changchun: IEEE, 2022. 644–648.
    [12] Hu TS, Fei YS. QELAR: A Q-learning-based energy-efficient and lifetime-aware routing protocol for underwater sensor networks. Proceedings of the 2008 IEEE International Performance, Computing and Communications Conference. Austin: IEEE, 2008. 247–255.
    [13] Fang ZR, Wang JJ, Jiang CX, et al. QLACO: Q-learning aided ant colony routing protocol for underwater acoustic sensor networks. Proceedings of the 2020 IEEE Wireless Communications and Networking Conference. Seoul: IEEE, 2020. 1–6.
    [14] Malajner M, Gleich D. Study of link quality indicator for possible usage in angle of arrival estimation. Proceedings of the 21 International Conference on Systems, Signals and Image Processing. Dubrovnik: IEEE, 2014. 195–198.
    [15] Haneef N, Chandavarkar BR. Dependency of routing protocol on pH and SNR in underwater communications. Proceedings of the 12th International Conference on Computing Communication and Networking Technologies. Kharagpur: IEEE, 2021. 1–7.
    [16] Deng ZC, Yu X, Zhu ZB, et al. Adaptive kalman filter based single beacon underwater tracking with unknown effective sound velocity. Proceedings of the 8th IEEE International Conference on Underwater System Technology: Theory and Applications. Wuhan: IEEE, 2018. 1–5.
    [17] Brekhovskikh LM, Lysanov YP. Fundamentals of Ocean Acoustics. 3rd ed., New York: Springer-Verlag, 2003.
    [18] Bhandarkar AB, Jayaweera SK. Optimal trajectory learning for UAV-mounted mobile base stations using RL and greedy algorithms. Proceedings of the 17th International Conference on Wireless and Mobile Computing, Networking and Communications. Bologna: IEEE, 2021. 13–18.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

廖学文,耿烜.基于Q学习的蚁群优化水声网络协议.计算机系统应用,2023,32(9):272-279

复制
分享
文章指标
  • 点击次数:562
  • 下载次数: 1529
  • HTML阅读次数: 1010
  • 引用次数: 0
历史
  • 收稿日期:2023-03-03
  • 最后修改日期:2023-04-04
  • 在线发布日期: 2023-07-21
文章二维码
您是第11417702位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号