基于LSTM的眼动行为识别及人机交互应用
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

广东省教育厅特色创新类(自然科学)项目(2017KTSCX207)


Eye Movement Recognition and Its Human-Computer Interaction Application Based on LSTM
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    眼动交互在人机交互领域中有着广泛的应用前景,针对传统的眼动交互传感设备具有普遍侵入性,校准过程复杂且价格昂贵,普通单目摄像头传感器分辨率低等问题.提出一种基于前置摄像头视频源,使用方向梯度直方图(HOG)特征+SVM+LSTM神经网络的眼动行为识别方法,进而实现了简单的人机交互应用.该方法首先定位和跟踪人脸,在人脸对齐操作后依据4个眼角关键点的坐标获取双眼区域,使用SVM模型判断眼睛的睁闭眼及非眨眼状态,再分析相邻帧之间眼球中心的位置粗略判断眼动情况,将可疑的有意眼势帧间差分视频序列输入到LSTM网络中进行预测,输出眼动行为识别结果,进而触发计算机命令完成交互.经过自制数据样本集20 000个样本(其中约10%为负样本)测试,本文方法动态眨眼识别准确率优于95%,眼动行为预测准确率为99.3%.

    Abstract:

    Eye-movement interaction has a broad application prospect in the field of human-computer interaction. Aiming at the problems of traditional eye-movement interaction sensors, such as universal intrusiveness, complex calibration process and high price, low resolution of common monocular camera sensors, etc., an eye movement recognition method based on front-facing camera video using directional gradient histogram (HOG) features + SVM + LSTM neural network, and a simple human-computer interaction application are proposed in this study. Firstly, the region of eyes are localized and tracked after face alignment. Secondly, the open-close and non-blinking state of the eyes is judged by the SVM model. Then, the position of eye center between adjacent frames is analyzed to roughly judge the eye movements, and the suspicious interframe difference video sequence of intentional eye position is obtained, which is the input of the LSTM network for prediction, and then trigger computer commands to complete the interaction. Through the self-made data sample set (about 10% of which are negative samples), the accuracy of dynamic blink recognition is better than 95%, and the accuracy of eye movement behavior prediction is 99.3%.

    参考文献
    相似文献
    引证文献
引用本文

黄君浩,贺辉.基于LSTM的眼动行为识别及人机交互应用.计算机系统应用,2020,29(3):206-212

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2019-09-03
  • 最后修改日期:2019-11-04
  • 录用日期:
  • 在线发布日期: 2020-03-02
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号