Static Gesture Recognition Based on Residual Double Attention Module and Cross-level Feature Fusion
CSTR:
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [28]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    To solve the problems of missing feature extraction by convolutional neural network and insufficient multi-feature extraction of a gesture, this study proposes a static gesture recognition method based on a residual double attention module and a cross-level feature fusion module. The designed residual double attention module can enhance the low-level features extracted by a ResNet50 network, effectively learn the key information, update the weight, and improve the attention to high-level features. Then, the cross-level feature fusion module fuses the high-level and low-level features in different stages to enrich the semantic and location information between different levels in the high-level feature map. Finally, the Softmax classifier of the fully connected layer is used to classify and recognize the gesture image. The experiment is carried out on the American sign language (ASL) dataset. The average recognition accuracy is 99.68%, which is 2.52% higher than that of the basic ResNet50 network. The results show that the proposed method can fully extract and reuse gesture features and effectively improve the recognition accuracy of gesture images.

    Reference
    [1] Dalal N, Triggs B. Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). San Diego: IEEE, 2005. 886–893.
    [2] Lowe DG. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110. [doi: 10.1023/B:VISI.0000029664.99615.94
    [3] 吴晓雨, 杨成, 冯琦. 基于Kinect的手势识别算法研究及应用. 计算机应用与软件, 2015, 32(7): 173–176, 276. [doi: 10.3969/j.issn.1000-386x.2015.07.042
    [4] 唐文权, 徐武, 文聪, 等. 复杂背景下基于肤色检测的动态手势分割与识别. 科学技术与工程, 2019, 19(33): 330–335. [doi: 10.3969/j.issn.1671-1815.2019.33.049
    [5] 李航, 厉丹, 朱晨, 等. 基于卷积神经网络的图像识别系统. 电脑知识与技术, 2020, 16(10): 196–197, 200
    [6] Pigou L, Dieleman S, Kindermans PJ, et al. Sign language recognition using convolutional neural networks. Proceedings of ECCV 2014 Workshops. Cham: Springer, 2015. 915–922.
    [7] Garcia B, Viesca SA. Real-time American sign language recognition with convolutional neural networks. Convolutional Neural Networks for Visual Recognition, 2016, 2: 225–232
    [8] Jain V, Jain A, Chauhan A, et al. American sign language recognition using support vector machine and convolutional neural network. International Journal of Information Technology, 2021, 13(3): 1193–1200. [doi: 10.1007/s41870-021-00617-x
    [9] Bheda V, Radpour D. Using deep convolutional networks for gesture recognition in American sign language. arXiv: 1710.06836, 2017.
    [10] Bantupalli K, Xie Y. American sign language recognition using deep learning and computer vision. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data). Seattle: IEEE, 2018. 4896–4899.
    [11] Yang L, Qi Z, Liu ZH, et al. An embedded implementation of CNN-based hand detection and orientation estimation algorithm. Machine Vision and Applications, 2019, 30(6): 1071–1082. [doi: 10.1007/s00138-019-01038-4
    [12] 吴晓凤, 张江鑫, 徐欣晨. 基于Faster R-CNN的手势识别算法. 计算机辅助设计与图形学学报, 2018, 30(3): 468–476
    [13] 陈影柔, 田秋红, 杨慧敏, 等. 基于多特征加权融合的静态手势识别. 计算机系统应用, 2021, 30(2): 20–27. [doi: 10.15888/j.cnki.csa.007748
    [14] 赵文清, 孔子旭, 周震东, 等. 增强小目标特征的航空遥感目标检测. 中国图象图形学报, 2021, 26(3): 644–653. [doi: 10.11834/jig.190612
    [15] Woo S, Park J, Lee JY, et al. CBAM: Convolutional block attention module. Proceedings of Computer Vision—ECCV 2018. Cham: Springer, 2018. 3–19.
    [16] 吴若有, 王德兴, 袁红春, 等. 基于多分支全卷积神经网络的低照度图像增强. 激光与光电子学进展, 2020, 57(14): 141021
    [17] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018. 7132–7141.
    [18] 宋东情, 朱定局, 贺超. 基于多模型特征与精简注意力融合的图像分类. 计算机系统应用, 2021, 30(11): 210–216. [doi: 10.15888/j.cnki.csa.008153
    [19] Chong TW, Lee BG. American sign language recognition using leap motion controller with machine learning approach. Sensors, 2018, 18(10): 3554. [doi: 10.3390/s18103554
    [20] Bin LY, Huann GY, Yun LK. Study of convolutional neural network in recognizing static American sign language. Proceedings of 2019 IEEE International Conference on Signal and Image Processing Applications (ICSIPA). Kuala Lumpur: IEEE, 2019. 41–45.
    [21] Lee CKM, Ng KKH, Chen CH, et al. American sign language recognition and training method with recurrent neural network. Expert Systems with Applications, 2021, 167: 114403. [doi: 10.1016/j.eswa.2020.114403
    [22] Rivera-Acosta M, Ruiz-Varela JM, Ortega-Cisneros S, et al. Spelling correction real-time American sign language alphabet translation system based on YOLO network and LSTM. Electronics, 2021, 10(9): 1035. [doi: 10.3390/electronics10091035
    [23] Sharma S, Kumar K. ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks. Multimedia Tools and Applications, 2021, 80(17): 26319–26331. [doi: 10.1007/s11042-021-10768-5
    [24] Purkaystha B, Datta T, Islam MS. Bengali handwritten character recognition using deep convolutional neural network. Proceedings of 2017 20th International Conference of Computer and Information Technology (ICCIT). Dhaka: IEEE, 2017. 1–5.
    [25] Ahmed S, Tabsun F, Reyadh AS, et al. Bengali handwritten alphabet recognition using deep convolutional neural network. Proceedings of 2019 5th International Conference on Computer, Communication, Chemical, Materials and Electronic Engineering (IC4ME2). Rajshahi: IEEE, 2019. 1–4.
    [26] Chatterjee S, Dutta RK, Ganguly D, et al. Bengali handwritten character classification using transfer learning on deep convolutional network. Proceedings of 11th International Conference on Intelligent Human Computer Interaction. Cham: Springer, 2020. 138–148.
    [27] Hasan M, Srizon AY, Hasan AM. Classification of Bengali sign language characters by applying a novel deep convolutional neural network. Proceedings of 2020 IEEE Region 10 Symposium (TENSYMP). Dhaka: IEEE, 2020. 1303–1306.
    [28] Hossain S, Sarma D, Mittra T, et al. Bengali hand sign gestures recognition using convolutional neural network. Proceedings of 2020 2nd International Conference on Inventive Research in Computing Applications (ICIRCA). Coimbatore: IEEE, 2020. 636–641.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

吴佳璐,田秋红,岳金鸿.基于残差双注意力与跨级特征融合模块的静态手势识别.计算机系统应用,2022,31(11):111-119

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:February 12,2022
  • Revised:March 14,2022
  • Online: July 07,2022
Article QR Code
You are the first990361Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063