Robotic Path Planning Integrating Improved Ant Colony Optimization and Dynamic Q-learning
CSTR:
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [20]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    When the basic Q-learning algorithm is applied to path planning, the randomness of action selection makes the early search efficiency of the algorithm low and the planning time-consuming, and even a complete and feasible path cannot be found. Therefore, a path planning algorithm of robots based on improved ant colony optimization (ACO) and dynamic Q-learning fusion is proposed. The pheromone increment mechanism of the elite ant model and sorting ant model is used, and a new pheromone increment updating method is designed to improve the exploration efficiency of robots. The pheromone matrix of the improved ant colony optimization algorithm is used to assign values to the Q table, so as to reduce the ineffective exploration of the robot at the initial stage. In addition, a dynamic selection strategy is designed to improve the convergence speed and the stability of the algorithm. Finally, different simulation experiments are carried out on two-dimensional static grid maps with different obstacle levels. The results show that the proposed method can effectively reduce the number of iterations and optimization time consumption in the optimization process.

    Reference
    [1] Josef S, Degani A. Deep reinforcement learning for safe local planning of a ground vehicle in unknown rough terrain. IEEE Robotics and Automation Letters, 2020, 5(4): 6748–6755. [doi: 10.1109/LRA.2020.3011912
    [2] 闫皎洁, 张锲石, 胡希平. 基于强化学习的路径规划技术综述. 计算机工程, 2021, 47(10): 16–25
    [3] 张荣霞, 武长旭, 孙同超, 等. 深度强化学习及在路径规划中的研究进展. 计算机工程与应用, 2021, 57(19): 44–56
    [4] Viswanathan S, Ravichandran KS, Tapas AM, et al. An intelligent gain based ant colony optimisation method for path planning of unmanned ground vehicles. Defence Science Journal, 2019, 69(2): 167–172. [doi: 10.14429/dsj.69.12509
    [5] Jiang L, Huang HY, Ding ZH. Path planning for intelligent robots based on deep Q-learning with experience replay and heuristic knowledge. IEEE/CAA Journal of Automatica Sinica, 2020, 7(4): 1179–1189. [doi: 10.1109/JAS.2019.1911732
    [6] 刘建伟, 高峰, 罗雄麟. 基于值函数和策略梯度的深度强化学习综述. 计算机学报, 2019, 42(6): 1406–1438. [doi: 10.11897/SP.J.1016.2019.01406
    [7] Tan B, Peng YY, Lin JG. A local path planning method based on Q-learning. Proceedings of the 2021 International Conference on Signal Processing and Machine Learning. Stanford: ACM, 2021. 80–84.
    [8] 马向华, 张谦. 改进蚁群算法在机器人路径规划上的研究. 计算机工程与应用, 2021, 57(5): 210–215
    [9] Low ES, Ong P, Cheah KC. Solving the optimal path planning of a mobile robot using improved Q-learning. Robotics and Autonomous Systems, 2019, 115: 143–161. [doi: 10.1016/j.robot.2019.02.013
    [10] 毛国君, 顾世民. 改进的Q-learning算法及其在路径规划中的应用. 太原理工大学学报, 2021, 52(1): 91–97
    [11] 徐晓苏, 袁杰. 基于改进强化学习的移动机器人路径规划方法. 中国惯性技术学报, 2019, 27(3): 314–320
    [12] Pei M, An H, Liu B, et al. An improved dyna-Q algorithm for mobile robot path planning in unknown dynamic environment. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2022, 52(7): 4415–4425. [doi: 10.1109/TSMC.2021.3096935
    [13] Hao B, Du H, Zhao JS, et al. A path-planning approach based on potential and dynamic Q-learning for mobile robots in unknown environment. Computational Intelligence and Neuroscience, 2022, 2022: 2540546
    [14] Chang L, Shan L, Jiang C, et al. Reinforcement based mobile robot path planning with improved dynamic window approach in unknown environment. Autonomous Robots, 2021, 45(1): 51–76. [doi: 10.1007/s10514-020-09947-4
    [15] 王晓燕, 杨乐, 张宇, 等. 基于改进势场蚁群算法的机器人路径规划. 控制与决策, 2018, 33(10): 1775–1781
    [16] 张恒, 何丽, 袁亮, 等. 基于改进双层蚁群算法的移动机器人路径规划. 控制与决策, 2022, 37(2): 303–313. [doi: 10.13195/j.kzyjc.2020.0610
    [17] 田晓航, 霍鑫, 周典乐, 等. 基于蚁群信息素辅助的Q学习路径规划算法. 控制与决策, 2022.
    [18] 石立培. 基于改进Q学习的智能车辆动态路径规划算法的研究[硕士学位论文]. 秦皇岛: 燕山大学, 2018.
    [19] 柯逸. 基于蚁群Q-learning算法的移动机器人路径规划[硕士学位论文]. 北京: 中国矿业大学, 2022.
    [20] Maoudj A, Hentout A. Optimal path planning approach based on Q-learning algorithm for mobile robots. Applied Soft Computing, 2020, 97: 106796. [doi: 10.1016/j.asoc.2020.106796
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

薛颂东,余欢.改进蚁群与动态Q学习融合的机器人路径规划.计算机系统应用,2023,32(8):189-197

Copy
Share
Article Metrics
  • Abstract:636
  • PDF: 1883
  • HTML: 941
  • Cited by: 0
History
  • Received:January 08,2023
  • Revised:February 09,2023
  • Online: May 22,2023
Article QR Code
You are the first990421Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063