Ant Colony Optimization Based on Q-learning for Underwater Acoustic Network Protocol
CSTR:
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [18]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    To solve the problems such as high data transmission delay and weak dynamic adaptability of underwater acoustic communication, this study proposes an intelligent underwater acoustic network routing protocol based on Q-learning ant colony optimization (QACO). The protocol includes routing behavior and intelligent decision. In the route discovery and maintenance phase, the construction of the network topology environment and information exchange among nodes as well as the network maintenance are carried out by intelligent NetAnts. In the Q-learning phase, the node energy and depth and network transmission delay learning characteristics are quantified as discount factors and learning rates to extend the network lifecycle and reduce system energy consumption and delay. Finally, simulations are carried out through the underwater acoustic network environment, and the experimental results show that QACO outperforms the Q-learning aided ant colony routing protocol (QLACO), Q-learning-based energy-efficient and lifetime-aware routing protocol (QELAR), and depth-based routing (DBR) algorithm in terms of energy consumption, delay, and network lifecycle.

    Reference
    [1] Zhang RX, Ma XL, Wang DQ, et al. Adaptive coding and bit-power loading algorithms for underwater acoustic transmissions. IEEE Transactions on Wireless Communications, 2021, 20(9): 5798–5811. [doi: 10.1109/TWC.2021.3070363
    [2] Doosti-Aref A, Ebrahimzadeh A. Adaptive relay selection and power allocation for OFDM cooperative underwater acoustic systems. IEEE Transactions on Mobile Computing, 2018, 17(1): 1–15
    [3] Chen XK, Yang HJ, Wang QM, et al. Network coding-based relay selection and power distribution for underwater cooperative communication networks. Proceedings of the 2020 International Conference on Wireless Communications and Signal Processing. Nanjing: IEEE, 2020. 596–601.
    [4] Xie P, Cui HJ, Lao L. VBF: Vector-based forwarding protocol for underwater sensor networks. Proceedings of the 5th International Conference on Research in Networking. Coimbra: Springer, 2006. 1216–1221.
    [5] Hwang D, Kim D. DFR: Directional flooding-based routing protocol for underwater sensor networks. Proceedings of the OCEANS 2008. Quebec City: IEEE, 2008. 1–7.
    [6] Yan H, Shi ZJ, Cui JH. DBR: Depth-based routing for underwater sensor networks. Proceedings of the 7th International Conference on Research in Networking. Singapore: Springer, 2008. 72–86.
    [7] Yildiz HU, Gungor VC, Tavli B. Packet size optimization for lifetime maximization in underwater acoustic sensor networks. IEEE Transactions on Industrial Informatics, 2019, 15(2): 719–729. [doi: 10.1109/TII.2018.2841830
    [8] Nazareth P, Chandavarkar BR. Link and void aware routing protocol for underwater acoustic sensor networks. Proceedings of the 12th International Conference on Computing Communication and Networking Technologies. Kharagpur: IEEE, 2021. 1–6.
    [9] Shi YM, Rong ZH. Analysis of Q-learning like algorithms through evolutionary game dynamics. IEEE Transactions on Circuits and Systems II: Express Briefs, 2022, 69(5): 2463–2467. [doi: 10.1109/TCSII.2022.3161655
    [10] Dorigo M, Maniezzo V, Colorni A. Ant system optimization by a colony of cooperating agents.IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics). 1996, 26(1): 29–41.
    [11] Lyu L, Shen Y, Zhang SC. The advance of reinforcement learning and deep reinforcement learning. Proceedings of the 2022 IEEE International Conference on Electrical Engineering, Big Data and Algorithms. Changchun: IEEE, 2022. 644–648.
    [12] Hu TS, Fei YS. QELAR: A Q-learning-based energy-efficient and lifetime-aware routing protocol for underwater sensor networks. Proceedings of the 2008 IEEE International Performance, Computing and Communications Conference. Austin: IEEE, 2008. 247–255.
    [13] Fang ZR, Wang JJ, Jiang CX, et al. QLACO: Q-learning aided ant colony routing protocol for underwater acoustic sensor networks. Proceedings of the 2020 IEEE Wireless Communications and Networking Conference. Seoul: IEEE, 2020. 1–6.
    [14] Malajner M, Gleich D. Study of link quality indicator for possible usage in angle of arrival estimation. Proceedings of the 21 International Conference on Systems, Signals and Image Processing. Dubrovnik: IEEE, 2014. 195–198.
    [15] Haneef N, Chandavarkar BR. Dependency of routing protocol on pH and SNR in underwater communications. Proceedings of the 12th International Conference on Computing Communication and Networking Technologies. Kharagpur: IEEE, 2021. 1–7.
    [16] Deng ZC, Yu X, Zhu ZB, et al. Adaptive kalman filter based single beacon underwater tracking with unknown effective sound velocity. Proceedings of the 8th IEEE International Conference on Underwater System Technology: Theory and Applications. Wuhan: IEEE, 2018. 1–5.
    [17] Brekhovskikh LM, Lysanov YP. Fundamentals of Ocean Acoustics. 3rd ed., New York: Springer-Verlag, 2003.
    [18] Bhandarkar AB, Jayaweera SK. Optimal trajectory learning for UAV-mounted mobile base stations using RL and greedy algorithms. Proceedings of the 17th International Conference on Wireless and Mobile Computing, Networking and Communications. Bologna: IEEE, 2021. 13–18.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

廖学文,耿烜.基于Q学习的蚁群优化水声网络协议.计算机系统应用,2023,32(9):272-279

Copy
Share
Article Metrics
  • Abstract:542
  • PDF: 1507
  • HTML: 967
  • Cited by: 0
History
  • Received:March 03,2023
  • Revised:April 04,2023
  • Online: July 21,2023
Article QR Code
You are the first990356Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063