###
计算机系统应用:2018,27(12):234-239
本文二维码信息
码上扫一扫!
基于RGB-D视频的多模态手势识别
马正文1, 蔡坚勇1,2,3,4,5, 刘磊1, 欧阳乐峰1, 李楠1
(1.福建师范大学 光电与信息工程学院, 福州 350007;2.福建师范大学 医学光电科学与技术教育部重点实验室, 福州 350007;3.福建师范大学 福建省光子技术重点实验室, 福州 350007;4.福建师范大学 福建省光电传感应用工程技术研究中心, 福州 350007;5.福建师范大学 智能光电系统工程研究中心, 福州 350007)
Multimodal Gesture Recognition Based on RGB-D Video
MA Zheng-Wen1, CAI Jian-Yong1,2,3,4,5, LIU Lei1, OUYANG Le-Feng1, LI Nan1
(1.College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China;2.Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Normal University, Fuzhou 350007, China;3.Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou 350007, China;4.Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou 350007, China;5.Intelligent Optoelectronic Systems Research Center, Fujian Normal University, Fuzhou 350007, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 20次   下载 15
投稿时间:2018-05-02    修订日期:2018-05-24
中文摘要: 本文是对SKIG RGB-D多模态的孤立手势视频进行手势识别研究.首先将RGB和Depth两种单模态视频提取成图片的形式保存,然后采样成长度为32帧的手势序列分别输入到本文提出的稠密连接的3DCNN组件学习短期的时空域特征,然后将提取的时空域特征输入到卷积GRU网络进行长期的时空域特征学习,最终对单模态训练好的网络进行多模态融合,提升网络识别准确率.本文在SKIG数据集上取得了99.07%的识别准确率,达到了极高的准确率,证明了本文提出的网络模型的有效性.
Abstract:In this study, the gesture recognition based on SKIG RGB-D multimodal isolated gesture video is studied. The RGB and depth videos are extracted into the form of images. Then the sampled 32 frames from images are input to the densely connected 3DCNN component to learn short-term spatiotemporal features, after that the features input to the convolutional GRU to learn long-term spatiotemporal features. Finally, the trained networks for single modal are used to multimodal fusion to improve the recognition accuracy. 99.07% recognition accuracy is obtained on the SKIG dataset, which achieves high accuracy and proves the validity of the network model proposed in this study.
文章编号:     中图分类号:    文献标志码:
基金项目:福建省自然科学基金(2017J01744)
引用文本:
马正文,蔡坚勇,刘磊,欧阳乐峰,李楠.基于RGB-D视频的多模态手势识别.计算机系统应用,2018,27(12):234-239
MA Zheng-Wen,CAI Jian-Yong,LIU Lei,OUYANG Le-Feng,LI Nan.Multimodal Gesture Recognition Based on RGB-D Video.COMPUTER SYSTEMS APPLICATIONS,2018,27(12):234-239

用微信扫一扫

用微信扫一扫