Deep Reinforcement Learning for Object Detection Based on Improved Reward Mechanism
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [20]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    To improve the detection accuracy and speed of deep reinforcement learning object detection models, modifications are made to traditional models. To address inadequate feature extraction, a VGG16 feature extraction module integrated with a channel attention mechanism is introduced as the state input for reinforcement learning, enabling a more comprehensive capture of key information in images. To address inaccurate evaluation caused by relying solely on the intersection over union as a reward, an improved reward mechanism that also considers the distance between the center points and the aspect ratio of the ground truth box and the predicted box is employed, making the reward more reasonable. To accelerate the convergence of the training process and enhance the objectivity of the agent’s evaluation of current states and actions, the Dueling DQN algorithm is used for training. After conducting experiments on the PASCAL VOC2007 and PASCAL VOC2012 datasets, experimental results show that the detection model only needs 4–10 candidate boxes to detect the target. Compared with Caicedo-RL, the accuracy is improved by 9.8%, and the mean intersection over union between the predicted and ground truth boxes is increased by 5.6%.

    Reference
    [1] Lowe DG. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110.
    [2] Dalal N, Triggs B. Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego: IEEE, 2005. 886–893.
    [3] Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014. 580–587.
    [4] Girshick R. Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). Santiago: IEEE, 2015. 1440–1448.
    [5] Ren SQ, He KM, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149.
    [6] He KM, Gkioxari G, Dollár P, et al. Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017. 2980–2988.
    [7] Cai ZW, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018. 6154–6162.
    [8] Yang J, Wang LQ. Feature fusion and enhancement for single shot multibox detector. Proceedings of the 2019 Chinese Automation Congress (CAC). Hangzhou: IEEE, 2019. 2766–2770.
    [9] Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016. 779–788.
    [10] Lin TY, Goyal P, Girshick R, et al. Focal loss for dense object detection. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017. 2999–3007.
    [11] Caicedo JC, Lazebnik S. Active object localization with deep reinforcement learning. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). Santiago: IEEE, 2015. 2488–2496.
    [12] Bellver M, Giró-i-Nieto X, Marqués F, et al. Hierarchical object detection with deep reinforcement learning. arXiv: 1611.03718, 2016.
    [13] Jie ZQ, Liang XD, Feng JS, et al. Tree-structured reinforcement learning for sequential object localization. Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: Curran Associates Inc., 2016. 127–135.
    [14] Xu N, Huo CL, Zhang X, et al. AHDet: A dynamic coarse-to-fine gaze strategy for active object detection. Neurocomputing, 2022, 491: 522–532.
    [15] Ayle M, Tekli J, El-Zini J, et al. Bar—A reinforcement learning agent for bounding-box automated refinement. Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020. 2561–2568.
    [16] König J, Malberg S, Martens M, et al. Multi-stage reinforcement learning for object detection. Proceedings of the 2019 Computer Vision Conference. Las Vegas: Springer, 2019. 178–191.
    [17] Zhou M, Wang RJ, Xie CJ, et al. ReinforceNet: A reinforcement learning embedded object detection framework with region selection network. Neurocomputing, 2021, 443: 369–379.
    [18] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018. 7132–7141.
    [19] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations. San Diego: ICLR, 2015.
    [20] Wang ZY, Schaul T, Hessel M, et al. Dueling network architectures for deep reinforcement learning. Proceedings of the 33rd International Conference on Machine Learning. New York: JMLR.org, 2016. 1995–2003.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

陈盈君,武月,刘力铭.基于改进奖励机制的深度强化学习目标检测.计算机系统应用,2024,33(10):106-114

Copy
Share
Article Metrics
  • Abstract:250
  • PDF: 1584
  • HTML: 680
  • Cited by: 0
History
  • Received:March 28,2024
  • Revised:May 06,2024
  • Online: August 28,2024
Article QR Code
You are the first987776Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063