AIPUB归智期刊联盟

微信公众号

网站二维码

2025年4月4日 18:29 星期五
首页 > 过刊浏览>2021年第30卷第1期 >221-227. DOI:10.15888/j.cnki.csa.007706
PDF HTML阅读 XML下载 导出引用 引用提醒
结合C3D与光流法的微表情自动识别
作者:
基金项目:

国家自然科学基金(61763002)


Automatic Recognition of Microexpression Based on C3D and Optical Flow
Author:
  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [31]
    • |
  • 相似文献 [20]
    • |
  • 引证文献
    • | |
  • 文章评论
    摘要:

    由于微表情动作幅度小且持续时间短, 使其识别难度大. 针对此问题, 提出一个结合三维卷积神经网络(3D Convolutional neural network, C3D)和光流法的微表情识别方法. 所提出的方法先用光流法从微表情视频中提取出包含动态特征的光流图像系列, 然后将得到的光流图像系列与原始灰度图像序列一起输入到C3D网络, 由C3D进一步提取微表情在时域和空域上的特征. 在开放数据集CASMEⅡ上进行了模拟实验, 实验表明本文所提出的方法对微表情的识别准确率达到67.53%, 优于现有方法.

    Abstract:

    It is difficult to recognize microexpression because of its small range and short duration. To solve this problem, a micro expression recognition method based on 3D Convolutional neural network (C3D) and optical flow method is proposed. We first extract a series of optical flow images with dynamic features from the microexpression video by optical flow method, then input the obtained series of optical flow images with the original gray-scale image sequences into the C3D network, and then extract the features of micro expression in the time and space domain by C3D. Simulation experiments on the open data set CASMEⅡ show that the recognition accuracy of the proposed method is 67.53%, which is better than the existing methods.

    参考文献
    [1] Corneanu CA, Simón MO, Cohn JF, et al. Survey on RGB, 3D, thermal, and multimodal approaches for facial expression recognition: History, trends, and affect-related applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(8): 1548–1568. [doi: 10.1109/TPAMI.2016.2515606
    [2] Russell TA, Chu E, Phillips ML. A pilot study to investigate the effectiveness of emotion recognition remediation in schizophrenia using the micro-expression training tool. British Journal of Clinical Psychology, 2006, 45(4): 579–583. [doi: 10.1348/014466505X90866
    [3] Pfister T, Li XB, Zhao GY, et al. Recognising spontaneous facial micro-expressions. Proceedings of 2011 International Conference on Computer Vision. Barcelona, Spain. 2011. 1449–1456.
    [4] Wang YD, See J, Phan RCW, et al. LBP with six intersection points: Reducing redundant information in LBP-TOP for micro-expression recognition. Proceedings of the 12th Asian Conference on Computer Vision. Singapore. 2015. 525–537.
    [5] Wang YD, See J, Phan RCW, et al. Efficient spatio-temporal local binary patterns for spontaneous facial micro-expression recognition. PLoS One, 2015, 10(5): e0124674. [doi: 10.1371/journal.pone.0124674
    [6] Huang XH, Zhao GY, Hong XP, et al. Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns. Neurocomputing, 2016, 175: 564–578. [doi: 10.1016/j.neucom.2015.10.096
    [7] Huang XH, Wang SJ, Zhao GY, et al. Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection. Proceedings of the IEEE International Conference on Computer Vision Workshop. Santiago, Chile. 2015. 1–9.
    [8] He JC, Hu JF, Lu X, et al. Multi-task mid-level feature learning for micro-expression recognition. Pattern Recognition, 2017, 66: 44–52. [doi: 10.1016/j.patcog.2016.11.029
    [9] Xu F, Zhang JP, Wang JZ. Microexpression identification and categorization using a facial dynamics map. IEEE Transactions on Affective Computing, 2017, 8(2): 254–267. [doi: 10.1109/TAFFC.2016.2518162
    [10] Liu YJ, Zhang JK, Yan WJ, et al. A main directional mean optical flow feature for spontaneous micro-expression recognition. IEEE Transactions on Affective Computing, 2016, 7(4): 299–310. [doi: 10.1109/TAFFC.2015.2485205
    [11] Liong ST, See J, Wong K, et al. Less is more: Micro-expression recognition from video using apex frame. Signal Processing: Image Communication, 2018, 62: 82–92. [doi: 10.1016/j.image.2017.11.006
    [12] Huang WB, Fan LJ, Harandi M, et al. Toward efficient action recognition: Principal backpropagation for training two-stream networks. IEEE Transactions on Image Processing, 2019, 28(4): 1773–1782. [doi: 10.1109/TIP.2018.2877936
    [13] Young T, Hazarika D, Poria S, et al. Recent trends in deep learning based natural language processing. IEEE Computational Intelligence Magazine, 2018, 13(3): 55–75. [doi: 10.1109/MCI.2018.2840738
    [14] Xiong W, Wu L, Alleva F, et al. The Microsoft 2017 conversational speech recognition system. Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing. Calgary, AB, Canada. 2018.5934–5938.
    [15] Patel D, Hong XP, Zhao GY. Selective deep features for micro-expression recognition. Proceedings of 2016 23rd International Conference on Pattern Recognition. Cancun, Mexico. 2016. 2258–2263.
    [16] Kim DH, Baddar WJ, Ro YM. Micro-expression recognition with expression-state constrained spatio-temporal feature representations. Proceedings of the 24th ACM International Conference on Multimedia. Amsterdam, the Netherlands. 2016. 382–386.
    [17] Peng M, Wang CY, Chen T, et al. Dual temporal scale convolutional neural network for micro-expression recognition. Frontiers in Psychology, 2017, 8: 1745. [doi: 10.3389/fpsyg.2017.01745
    [18] Khor HQ, See J, Phan RCW, et al. Enriched long-term recurrent convolutional network for facial micro-expression recognition. Proceedings of 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition. Xi’an, China. 2018. 667–674.
    [19] Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3D convolutional networks. Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile. 2015. 4489–4497.
    [20] Barron JL, Fleet DJ, Beauchemin SS. Performance of optical flow techniques. International Journal of Computer Vision, 1994, 12(1): 43–77. [doi: 10.1007/BF01420984
    [21] Negahdaripour S. Revised definition of optical flow: Integration of radiometric and geometric cues for dynamic scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(9): 961–979. [doi: 10.1109/34.713362
    [22] Yan WJ, Li XB, Wang SJ, et al. CASME II: An improved spontaneous micro-expression database and the baseline evaluation. PLoS One, 2014, 9(1): e86041. [doi: 10.1371/journal.pone.0086041
    [23] Ji SW, Xu W, Yang M, et al. 3D convolutional neural networks for human action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 221–231. [doi: 10.1109/TPAMI.2012.59
    [24] Farnebäck G. Two-frame motion estimation based on polynomial expansion. Proceedings of the 13th Scandinavian Conference on Image Analysis. Halmstad, Sweden. 2003.363–370.
    [25] Li XB, Pfister T, Huang XH, et al. A spontaneous micro-expression database: Inducement, collection and baseline. Proceedings of 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). Shanghai, China. 2013. 1–6.
    [26] Yan WJ, Wu Q, Liu YJ, et al. CASME database: A dataset of spontaneous micro-expressions collected from neutralized faces. Proceedings of 2013 10th IEEE International Conference and Workshops on Automatic face and Gesture Recognition (FG). Shanghai, China. 2013. 1–7.
    [27] Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA. 2014. 1867–1874.
    [28] Zhou ZH, Zhao GY, Pietikäinen M. Towards a practical lipreading system. Proceedings of CVPR 2011. Providence, RI, USA. 2011. 137–144.
    [29] Zhong Z, Zheng L, Zheng ZD, et al. CamStyle: A novel data augmentation method for person re-identification. IEEE Transactions on Image Processing, 2019, 28(3): 1176–1190. [doi: 10.1109/TIP.2018.2874313
    [30] Gargi U, Kasturi R, Strayer SH. Performance characterization of video-shot-change detection methods. IEEE Transactions on Circuits and Systems for Video Technology, 2000, 10(1): 1–13. [doi: 10.1109/76.825852
    [31] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. The 3rd International Conference on Learning Representations. San Diego, CA, USA. 2015. 1–5.
    引证文献
引用本文

何景琳,梁正友,孙宇,刘德志.结合C3D与光流法的微表情自动识别.计算机系统应用,2021,30(1):221-227

复制
分享
文章指标
  • 点击次数:1125
  • 下载次数: 2556
  • HTML阅读次数: 2558
  • 引用次数: 0
历史
  • 收稿日期:2020-05-01
  • 最后修改日期:2020-05-27
  • 在线发布日期: 2020-12-31
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会软件学报
您是第11275056位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码：100190
电话：010-62661041 传真： Email：csa (a) iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号