AIPUB归智期刊联盟

WeChat

Mobile website

2025-4-3- 11
Home > Archive>Volume 30, Issue 1, 2021 >221-227. DOI:10.15888/j.cnki.csa.007706
PDF HTML XML Export Cite reminder
Automatic Recognition of Microexpression Based on C3D and Optical Flow
CSTR:
[cstr]
Author:
  • Article
    • | |
  • Metrics
    • |
  • Reference [31]
    • |
  • Related [20]
    • | | |
  • Comments
    Abstract:

    It is difficult to recognize microexpression because of its small range and short duration. To solve this problem, a micro expression recognition method based on 3D Convolutional neural network (C3D) and optical flow method is proposed. We first extract a series of optical flow images with dynamic features from the microexpression video by optical flow method, then input the obtained series of optical flow images with the original gray-scale image sequences into the C3D network, and then extract the features of micro expression in the time and space domain by C3D. Simulation experiments on the open data set CASMEⅡ show that the recognition accuracy of the proposed method is 67.53%, which is better than the existing methods.

    Reference
    [1] Corneanu CA, Simón MO, Cohn JF, et al. Survey on RGB, 3D, thermal, and multimodal approaches for facial expression recognition: History, trends, and affect-related applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(8): 1548–1568. [doi: 10.1109/TPAMI.2016.2515606
    [2] Russell TA, Chu E, Phillips ML. A pilot study to investigate the effectiveness of emotion recognition remediation in schizophrenia using the micro-expression training tool. British Journal of Clinical Psychology, 2006, 45(4): 579–583. [doi: 10.1348/014466505X90866
    [3] Pfister T, Li XB, Zhao GY, et al. Recognising spontaneous facial micro-expressions. Proceedings of 2011 International Conference on Computer Vision. Barcelona, Spain. 2011. 1449–1456.
    [4] Wang YD, See J, Phan RCW, et al. LBP with six intersection points: Reducing redundant information in LBP-TOP for micro-expression recognition. Proceedings of the 12th Asian Conference on Computer Vision. Singapore. 2015. 525–537.
    [5] Wang YD, See J, Phan RCW, et al. Efficient spatio-temporal local binary patterns for spontaneous facial micro-expression recognition. PLoS One, 2015, 10(5): e0124674. [doi: 10.1371/journal.pone.0124674
    [6] Huang XH, Zhao GY, Hong XP, et al. Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns. Neurocomputing, 2016, 175: 564–578. [doi: 10.1016/j.neucom.2015.10.096
    [7] Huang XH, Wang SJ, Zhao GY, et al. Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection. Proceedings of the IEEE International Conference on Computer Vision Workshop. Santiago, Chile. 2015. 1–9.
    [8] He JC, Hu JF, Lu X, et al. Multi-task mid-level feature learning for micro-expression recognition. Pattern Recognition, 2017, 66: 44–52. [doi: 10.1016/j.patcog.2016.11.029
    [9] Xu F, Zhang JP, Wang JZ. Microexpression identification and categorization using a facial dynamics map. IEEE Transactions on Affective Computing, 2017, 8(2): 254–267. [doi: 10.1109/TAFFC.2016.2518162
    [10] Liu YJ, Zhang JK, Yan WJ, et al. A main directional mean optical flow feature for spontaneous micro-expression recognition. IEEE Transactions on Affective Computing, 2016, 7(4): 299–310. [doi: 10.1109/TAFFC.2015.2485205
    [11] Liong ST, See J, Wong K, et al. Less is more: Micro-expression recognition from video using apex frame. Signal Processing: Image Communication, 2018, 62: 82–92. [doi: 10.1016/j.image.2017.11.006
    [12] Huang WB, Fan LJ, Harandi M, et al. Toward efficient action recognition: Principal backpropagation for training two-stream networks. IEEE Transactions on Image Processing, 2019, 28(4): 1773–1782. [doi: 10.1109/TIP.2018.2877936
    [13] Young T, Hazarika D, Poria S, et al. Recent trends in deep learning based natural language processing. IEEE Computational Intelligence Magazine, 2018, 13(3): 55–75. [doi: 10.1109/MCI.2018.2840738
    [14] Xiong W, Wu L, Alleva F, et al. The Microsoft 2017 conversational speech recognition system. Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing. Calgary, AB, Canada. 2018.5934–5938.
    [15] Patel D, Hong XP, Zhao GY. Selective deep features for micro-expression recognition. Proceedings of 2016 23rd International Conference on Pattern Recognition. Cancun, Mexico. 2016. 2258–2263.
    [16] Kim DH, Baddar WJ, Ro YM. Micro-expression recognition with expression-state constrained spatio-temporal feature representations. Proceedings of the 24th ACM International Conference on Multimedia. Amsterdam, the Netherlands. 2016. 382–386.
    [17] Peng M, Wang CY, Chen T, et al. Dual temporal scale convolutional neural network for micro-expression recognition. Frontiers in Psychology, 2017, 8: 1745. [doi: 10.3389/fpsyg.2017.01745
    [18] Khor HQ, See J, Phan RCW, et al. Enriched long-term recurrent convolutional network for facial micro-expression recognition. Proceedings of 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition. Xi’an, China. 2018. 667–674.
    [19] Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3D convolutional networks. Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile. 2015. 4489–4497.
    [20] Barron JL, Fleet DJ, Beauchemin SS. Performance of optical flow techniques. International Journal of Computer Vision, 1994, 12(1): 43–77. [doi: 10.1007/BF01420984
    [21] Negahdaripour S. Revised definition of optical flow: Integration of radiometric and geometric cues for dynamic scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(9): 961–979. [doi: 10.1109/34.713362
    [22] Yan WJ, Li XB, Wang SJ, et al. CASME II: An improved spontaneous micro-expression database and the baseline evaluation. PLoS One, 2014, 9(1): e86041. [doi: 10.1371/journal.pone.0086041
    [23] Ji SW, Xu W, Yang M, et al. 3D convolutional neural networks for human action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 221–231. [doi: 10.1109/TPAMI.2012.59
    [24] Farnebäck G. Two-frame motion estimation based on polynomial expansion. Proceedings of the 13th Scandinavian Conference on Image Analysis. Halmstad, Sweden. 2003.363–370.
    [25] Li XB, Pfister T, Huang XH, et al. A spontaneous micro-expression database: Inducement, collection and baseline. Proceedings of 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). Shanghai, China. 2013. 1–6.
    [26] Yan WJ, Wu Q, Liu YJ, et al. CASME database: A dataset of spontaneous micro-expressions collected from neutralized faces. Proceedings of 2013 10th IEEE International Conference and Workshops on Automatic face and Gesture Recognition (FG). Shanghai, China. 2013. 1–7.
    [27] Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA. 2014. 1867–1874.
    [28] Zhou ZH, Zhao GY, Pietikäinen M. Towards a practical lipreading system. Proceedings of CVPR 2011. Providence, RI, USA. 2011. 137–144.
    [29] Zhong Z, Zheng L, Zheng ZD, et al. CamStyle: A novel data augmentation method for person re-identification. IEEE Transactions on Image Processing, 2019, 28(3): 1176–1190. [doi: 10.1109/TIP.2018.2874313
    [30] Gargi U, Kasturi R, Strayer SH. Performance characterization of video-shot-change detection methods. IEEE Transactions on Circuits and Systems for Video Technology, 2000, 10(1): 1–13. [doi: 10.1109/76.825852
    [31] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. The 3rd International Conference on Learning Representations. San Diego, CA, USA. 2015. 1–5.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

何景琳,梁正友,孙宇,刘德志.结合C3D与光流法的微表情自动识别.计算机系统应用,2021,30(1):221-227

Copy
Share
Article Metrics
  • Abstract:1123
  • PDF: 2555
  • HTML: 2557
  • Cited by: 0
History
  • Received:May 01,2020
  • Revised:May 27,2020
  • Online: December 31,2020
Article QR Code
Links:Journal of Software中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会
You are the first990404Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address：4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code：100190
Phone：010-62661041 Fax： Email：csa (a) iscas.ac.cn
Technical Support：Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063