码上扫一扫！
基于RGB-D视频的多模态手势识别
(1.福建师范大学 光电与信息工程学院, 福州 350007;2.福建师范大学 医学光电科学与技术教育部重点实验室, 福州 350007;3.福建师范大学 福建省光子技术重点实验室, 福州 350007;4.福建师范大学 福建省光电传感应用工程技术研究中心, 福州 350007;5.福建师范大学 智能光电系统工程研究中心, 福州 350007)
Multimodal Gesture Recognition Based on RGB-D Video
(1.College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China;2.Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Normal University, Fuzhou 350007, China;3.Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou 350007, China;4.Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou 350007, China;5.Intelligent Optoelectronic Systems Research Center, Fujian Normal University, Fuzhou 350007, China)
本文已被：浏览 1770次 下载 2436次
Received:May 02, 2018 Revised:May 24, 2018
中文摘要: 本文是对SKIG RGB-D多模态的孤立手势视频进行手势识别研究.首先将RGB和Depth两种单模态视频提取成图片的形式保存，然后采样成长度为32帧的手势序列分别输入到本文提出的稠密连接的3DCNN组件学习短期的时空域特征，然后将提取的时空域特征输入到卷积GRU网络进行长期的时空域特征学习，最终对单模态训练好的网络进行多模态融合，提升网络识别准确率.本文在SKIG数据集上取得了99.07%的识别准确率，达到了极高的准确率，证明了本文提出的网络模型的有效性.
Abstract:In this study, the gesture recognition based on SKIG RGB-D multimodal isolated gesture video is studied. The RGB and depth videos are extracted into the form of images. Then the sampled 32 frames from images are input to the densely connected 3DCNN component to learn short-term spatiotemporal features, after that the features input to the convolutional GRU to learn long-term spatiotemporal features. Finally, the trained networks for single modal are used to multimodal fusion to improve the recognition accuracy. 99.07% recognition accuracy is obtained on the SKIG dataset, which achieves high accuracy and proves the validity of the network model proposed in this study.
文章编号： 中图分类号： 文献标志码：
基金项目:福建省自然科学基金（2017J01744）
| Author Name | Affiliation | E-mail |
| MA Zheng-Wen | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China | |
| CAI Jian-Yong | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China
Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Normal University, Fuzhou 350007, China
Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou 350007, China
Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou 350007, China
Intelligent Optoelectronic Systems Research Center, Fujian Normal University, Fuzhou 350007, China | cjy@fjnu.edu.cn |
| LIU Lei | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China | |
| OUYANG Le-Feng | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China | |
| LI Nan | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China | |
| Author Name | Affiliation | E-mail |
| MA Zheng-Wen | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China | |
| CAI Jian-Yong | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China
Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Normal University, Fuzhou 350007, China
Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou 350007, China
Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou 350007, China
Intelligent Optoelectronic Systems Research Center, Fujian Normal University, Fuzhou 350007, China | cjy@fjnu.edu.cn |
| LIU Lei | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China | |
| OUYANG Le-Feng | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China | |
| LI Nan | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China | |
引用文本：
马正文,蔡坚勇,刘磊,欧阳乐峰,李楠.基于RGB-D视频的多模态手势识别.计算机系统应用,2018,27(12):234-239
MA Zheng-Wen,CAI Jian-Yong,LIU Lei,OUYANG Le-Feng,LI Nan.Multimodal Gesture Recognition Based on RGB-D Video.COMPUTER SYSTEMS APPLICATIONS,2018,27(12):234-239
![]()
