基于深度学习的伪造人脸检测技术综述
作者:
基金项目:

国家自然科学基金(62471124); 黑龙江省自然科学基金(LH2022F006); 黑龙江省教育科学规划重点项目(GJB1421114)


Review of Forged Face Detection Techniques Based on Deep Learning
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [85]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    近年来, 随着伪造人脸技术的快速发展, 通过伪造人脸技术合成的人脸已经非常逼真, 人眼很难鉴别, 部分不法分子对伪造人脸技术的非法应用已经对社会稳定、个人隐私造成了恶劣影响, 因此伪造人脸检测技术的重要性日益凸显. 本文系统地探讨了伪造人脸检测技术的现状, 主要从伪造人脸图像和伪造人脸视频的检测两个方面进行分析. 在伪造人脸图像检测方面, 重点讨论了基于图像空间域和频率域的方法、身份一致性检测以及人脸区域定位技术的应用. 在伪造人脸视频检测方面, 研究聚焦于时空特征融合、生理特征利用及视听信息的结合. 此外, 本文介绍了常用的评估指标, 系统分析了多种重要数据集, 包括其特点和适用场景. 同时还指出当前文献中的局限性, 例如对抗样本的鲁棒性不足、检测方法对新型伪造技术的适应性差等问题. 基于这些分析, 我们提出了未来可能的研究方向, 包括跨域检测技术的优化、新算法的探索及模型的可解释性研究. 本文不仅为研究者提供了对伪造人脸检测技术的全面了解, 也为后续研究指明了发展方向, 具有重要的理论价值和实际应用意义.

    Abstract:

    In recent years, as the forged face technology rapidly develops, the face synthesized has been extremely hard for the human eyes to identify, and the application of this technology by some criminals has badly threatened social stability and personal privacy, so the importance of forged face detection technology has become increasingly prominent. This review systematically discusses the current status of forged face detection technology, mainly from two aspects of forged face image detection and forged face video detection. In the aspect of forged face image detection, the methods based on the image spatial domain and frequency domain, identity consistency detection, and the application of face region localization technology are discussed. In the field of forged face video detection, the research focuses on the integration of spatio-temporal features, the utilization of physiological features, and the combination of audiovisual information. In addition, the study introduces the commonly used evaluation indicators and systematically analyzes a variety of important data sets, including their characteristics and application scenarios. At the same time, it also points out the limitations in the current literature, such as the lack of robustness of adversarial samples and the poor adaptability of detection methods to new forgery techniques. Based on these analyses, this study puts forward the possible research directions in the future, including the optimization of cross-domain detection technology, the exploration of new algorithms, and the study of the model interpretability. This review not only provides researchers with a comprehensive understanding of fake face detection technology but also points out the development direction for subsequent research, possessing high theoretical value and practical application significance.

    参考文献
    [1] Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal: ACM, 2014. 2672–2680.
    [2] 邬林桦. 当“眼见不再为实”, 全新风险如何应对? 解放日报. https://baijiahao.baidu.com/s?id=1791478272339787059&wfr=spider&for=pc. (2024-02-21).
    [3] 张传鑫. 基于解耦学习的非特定GAN图像检测逃避方法 [硕士学位论文]. 广州: 华南理工大学, 2022.
    [4] McCloskey S, Albright M. Detecting GAN-generated imagery using color cues. arXiv:1812.08247, 2018.
    [5] 汤桂花, 孙磊, 毛秀青, 等. 基于深度对齐网络的生成对抗网络伪造人脸检测. 计算机应用, 2021, 41(7): 1922–1927.
    [6] 朱新同, 唐云祁, 耿鹏志. 基于特征融合的篡改与深度伪造图像检测算法. 信息网络安全, 2021, 21(8): 70–81.
    [7] Zhou P, Han XT, Morariu VI, et al. Two-stream neural networks for tampered face detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu: IEEE, 2017. 1831–1839.
    [8] Zhao HQ, Wei TY, Zhou WB, et al. Multi-attentional DeepFake detection. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021. 2185–2194.
    [9] Luo YC, Zhang Y, Yan JC, et al. Generalizing face forgery detection with high-frequency features. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021. 16312–16321.
    [10] Shao R, Wu TX, Nie LQ, et al. DeepFake-adapter: Dual-level adapter for DeepFake detection. arXiv:2306.00863, 2023.
    [11] Huang BJ, Wang ZY, Yang JF, et al. Implicit identity driven DeepFake face swapping detection. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023. 4490–4499.
    [12] 戴昀书, 费建伟, 夏志华, 等. 局部相似度异常的强泛化性伪造人脸检测. 中国图象图形学报, 2023, 28(11): 3453–3470.
    [13] 冯才博, 刘春晓, 王昱烨, 等. 结合图像块比较与残差图估计的人脸伪造检测. 中国图象图形学报, 2024, 29(2): 457–467.
    [14] Tan CC, Zhao Y, Wei SK, et al. Learning on gradients: Generalized artifacts representation for GAN-generated images detection. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023. 12105–12114.
    [15] Tan CC, Liu H, Zhao Y, et al. Rethinking the up-sampling operations in CNN-based generative network for generalizable DeepFake detection. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2024. 28130–28139.
    [16] Guo Y, Zhen C, Yan PF. Controllable guide-space for generalizable face forgery detection. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV). Paris: IEEE, 2023. 20761–20770.
    [17] Yan ZY, Zhang Y, Fan YB, et al. UCF: Uncovering common features for generalizable DeepFake detection. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV). Paris: IEEE, 2023. 22355–22366.
    [18] Bai WM, Liu YF, Zhang ZP, et al. AUNet: Learning relations between action units for face forgery detection. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023. 24709–24719.
    [19] Hou Y, Guo Q, Huang YH, et al. Evading DeepFake detectors via adversarial statistical consistency. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023. 12271–12280.
    [20] Nguyen D, Mejri N, Singh IP, et al. LAA-Net: Localized artifact attention network for quality-agnostic and generalizable DeepFake detection. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2024. 17395–17405.
    [21] Yan ZY, Luo YH, Lyu S, et al. Transcending forgery specificity with latent space augmentation for generalizable DeepFake detection. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2024. 8984–8994.
    [22] Yao KL, Wang J, Diao BY, et al. Towards understanding the generalization of DeepFake detectors from a game-theoretical view. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV). Paris: IEEE, 2023. 2031–2041.
    [23] Kim DK, Kim K. Generalized facial manipulation detection with edge region feature extraction. Proceedings of the 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa: IEEE, 2022. 2784–2794.
    [24] Qian YY, Yin GJ, Sheng L, et al. Thinking in frequency: Face forgery detection by mining frequency-aware clues. Proceedings of the 16th European Conference on Computer Vision. Glasgow: Springer, 2020. 86–103.
    [25] Li JM, Xie HT, Li JH, et al. Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021. 6454–6563.
    [26] Wang GJ, Li W, Jiang Q, et al. Using grayscale frequency statistic to detect manipulated faces in wavelet-domain. Proceedings of the 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Melbourne: IEEE, 2021. 2986–2993.
    [27] 李颖, 边山, 王春桃, 等. 基于双流网络结构的深度伪造人脸的检测方法. 计算机科学, 2022, 49(11A): 220100106.
    [28] Jeong Y, Kim D, Ro Y, et al. FrePGAN: Robust DeepFake detection using frequency-level perturbations. Proceedings of the 36th AAAI Conference on Artificial Intelligence. AAAI, 2022. 1060–1068.
    [29] 马欣, 吉立新, 李邵梅. 基于多尺度Transformer融合多域信息的伪造人脸检测. 计算机科学, 2023, 50(10): 112–118.
    [30] Liu HG, Li XD, Zhou WB, et al. Spatial-phase shallow learning: Rethinking face forgery detection in frequency domain. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021. 772–781.
    [31] Wang Y, Yu K, Chen C, et al. Dynamic graph learning with content-guided spatial-frequency relation reasoning for DeepFake detection. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023. 7278–7287.
    [32] Li JM, Xie HT, Yu LY, et al. Wavelet-enhanced weakly supervised local feature learning for face forgery detection. Proceedings of the 30th ACM International Conference on Multimedia. Lisboa: IEEE, 2022. 1299–1308.
    [33] Wolter M, Blanke F, Heese R, et al. Wavelet-packets for DeepFake image analysis and detection. Machine Learning, 2022, 111(11): 4295–4327.
    [34] Song LC, Fang Z, Li XD, et al. Adaptive face forgery detection in cross domain. Proceedings of the 17th European Conference on Computer Vision. Tel Aviv: Springer, 2022. 467–484.
    [35] Wang JK, Wu ZX, Ouyang WH, et al. M2TR: Multi-modal multi-scale Transformers for DeepFake detection. Proceedings of the 2022 International Conference on Multimedia Retrieval. Newark: ACM, 2022. 615–623.
    [36] Dong JH, Xie XH. Visually maintained image disturbance against DeepFake face swapping. Proceedings of the 2021 IEEE International Conference on Multimedia and Expo (ICME). Shenzhen: IEEE, 2021. 1–6.
    [37] Wang Z, Guo YW, Zuo WM. DeepFake forensics via an adversarial game. IEEE Transactions on Image Processing, 2022, 31: 3541–3552.
    [38] Chen L, Zhang Y, Song YB, et al. Self-supervised learning of adversarial example: Towards good generalizations for DeepFake detection. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE, 2022. 18689–18698.
    [39] 王瑜, 方贤进, 杨高明, 等. 基于注意力掩码与特征提取的人脸伪造主动防御. 计算机应用. https://link.cnki.net/urlid/51.1307.TP.20240802.1428.006. (2024-08-05)[2024-11-15].
    [40] Dong XY, Bao JM, Chen DD, et al. Identity-driven DeepFake detection. arXiv:2012.03930, 2020.
    [41] Liu BP, Liu B, Ding M, et al. TI2Net: Temporal identity inconsistency network for DeepFake detection. Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa: IEEE, 2023. 4680–4689.
    [42] Chugh K, Gupta P, Dhall A, et al. Not made for each other- audio-visual dissonance-based DeepFake detection and localization. Proceedings of the 28th ACM International Conference on Multimedia. Seattle: ACM, 2020. 439–447.
    [43] Dong XY, Bao JM, Chen DD, et al. Protecting celebrities from DeepFake with identity consistency Transformer. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE, 2022. 9458–9468.
    [44] Chu BL, You WK, Yang Z, et al. Protecting world leader using facial speaking pattern against DeepFakes. IEEE Signal Processing Letters, 2022, 29: 2078–2082.
    [45] Boháček M, Farid H. Protecting world leaders against deep fakes using facial, gestural, and vocal mannerisms. Proceedings of the 2022 National Academy of Sciences of the United States of America, 2022, 119(48): e2216035119.
    [46] Stehouwer J, Dang H, Liu F, et al. On the detection of digital face manipulation. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, 2020. 5780–5789.
    [47] Chen P, Liu J, Liang T, et al. DLFMNet: End-to-end detection and localization of face manipulation using multi-domain features. Proceedings of the 2021 IEEE International Conference on Multimedia and Expo. Shenzhen: IEEE, 2021. 1–6.
    [48] Guo X, Liu XH, Ren ZY, et al. Hierarchical fine-grained image forgery detection and localization. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023. 3155–3165.
    [49] Guillaro F, Cozzolino D, Sud A, et al. TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023. 20606–20615.
    [50] Sun ZK, Han YJ, Hua ZY, et al. Improving the efficiency and robustness of DeepFakes detection through precise geometric features. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021. 3608–3617.
    [51] 朱春陶, 尹承禧, 张博林, 等. 基于多域时序特征挖掘的伪造人脸检测方法. 网络与信息安全学报, 2023, 9(3): 123–134.
    [52] Fei JW, Xia ZH, Yu PP, et al. Exposing AI-generated videos with motion magnification. Multimedia Tools and Applications, 2021, 80(20): 30789–30802.
    [53] Zhang DC, Li CY, Lin FZ, et al. Detecting DeepFake videos with temporal dropout 3DCNN. Proceedings of the 30th International Joint Conference on Artificial Intelligence. Montreal: IJCAI, 2021. 1288–1294.
    [54] Zhang DC, Lin FZ, Hua YY, et al. DeepFake video detection with spatiotemporal dropout Transformer. Proceedings of the 30th ACM International Conference on Multimedia. Lisboa: ACM, 2022. 5833–5841.
    [55] Xu YT, Liang J, Jia GY, et al. TALL: Thumbnail layout for DeepFake video detection. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV). Paris: IEEE, 2023. 22601–22611.
    [56] 祝恺蔓, 徐文博, 卢伟, 等. 多关键帧特征交互的人脸篡改视频检测. 中国图象图形学报, 2022, 27(1): 188–202.
    [57] 程燕. 基于关键帧与时空特征融合的人脸伪造检测. 计算机科学, 2024, 51(11): 191–197.
    [58] Gu ZH, Yao TP, Chen Y, et al. Hierarchical contrastive inconsistency learning for DeepFake video detection. Proceedings of the 17th European Conference on Computer Vision. Tel Aviv: Springer, 2022. 596–613.
    [59] Gu ZH, Yao TP, Chen Y, et al. Region-aware temporal inconsistency learning for DeepFake video detection. Proceedings of the 31st International Joint Conference on Artificial Intelligence. Vienna: IJCAI, 2022. 920–926.
    [60] Hu J, Liao X, Liang JW, et al. FInfer: Frame inference-based DeepFake detection for high-visual-quality videos. Proceedings of the 36th AAAI Conference on Artificial Intelligence. AAAI, 2022. 951–959.
    [61] Gu ZH, Chen Y, Yao TP, et al. Delving into the local: Dynamic inconsistency learning for DeepFake video detection. Proceedings of the 36th AAAI Conference on Artificial Intelligence. AAAI, 2022. 744–752.
    [62] Choi J, Kim T, Jeong Y, et al. Exploiting style latent flows for generalizing DeepFake video detection. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2024. 1133–1143.
    [63] Tan LF, Wang YH, Wang JF, et al. DeepFake video detection via facial action dependencies estimation. Proceedings of the 37th AAAI Conference on Artificial Intelligence. Washington: IEEE, 2023. 5276–5284.
    [64] Li YZ, Chang MC, Lyu SW. In Ictu Oculi: Exposing AI created fake videos by detecting eye blinking. Proceedings of the 2018 IEEE International Workshop on Information Forensics and Security. Hong Kong: IEEE, 2018. 1–7.
    [65] Haliassos A, Vougioukas K, Petridis S, et al. Lips don’t lie: A generalisable and robust approach to face forgery detection. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2020. 5037–5047.
    [66] Agarwal S, Farid H. Detecting deep-fake videos from aural and oral dynamics. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021. 981–989.
    [67] Haliassos A, Mira R, Petridis S, et al. Leveraging real talking faces via self-supervision for robust forgery detection. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022. 14930–14942.
    [68] Hernandez-Ortega J, Tolosana R, Fiérrez J, et al. DeepFakesON-Phys: DeepFakes detection based on heart rate estimation. Proceedings of the 2021 Workshop on Artificial Intelligence Safety (SafeAI 2021) Co-located with the 35th AAAI Conference on Artificial Intelligence. AAAI, 2021.
    [69] Chen ML, Liao X, Wu M. PulseEdit: Editing physiological signals in facial videos for privacy protection. IEEE Transactions on Information Forensics and Security, 2022, 17: 457–471.
    [70] Lin JY, Zhou WB, Liu HG, et al. Lip forgery video detection via multi-phoneme selection. Proceedings of the 2021 International Workshop on Safety and Security of Deep Learning. New York, 2021.
    [71] Wu JH, Zhu Y, Jiang XB, et al. Local attention and long-distance interaction of rPPG for DeepFake detection. The Visual Computer, 2023, 40(2): 1083–1094.
    [72] Yu ZT, Cai RZ, Li Z, et al. Benchmarking joint face spoofing and forgery detection with visual and physiological cues. IEEE Transactions on Dependable and Secure Computing, 2024, 21(5): 4327–4342.
    [73] Kalin S, Bhawna P, Abhinav D. Visual representations of physiological signals for fake video detection. arXiv:2207.08380, 2022.
    [74] Feng C, Chen ZY, Owens A. Self-supervised video forensics by audio-visual anomaly detection. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023. 10491–10503.
    [75] Oorloff T, Koppisetti S, Bonettini N, et al. AVFF: Audio-visual feature fusion for video DeepFake detection. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2024. 27092–27102.
    [76] Mittal T, Bhattacharya U, Chandra R, et al. Emotions don’t lie: An audio-visual DeepFake detection method using affective cues. Proceedings of the 28th ACM International Conference on Multimedia. Seattle: ACM, 2020. 2823–2832.
    [77] Bohacek M, Farid H. Lost in translation: Lip-sync DeepFake detection from audio-video mismatch. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2024. 4315–4323.
    [78] Cheng H, Guo YY, Wang TY, et al. Voice-face homogeneity tells DeepFake. ACM Transactions on Multimedia Computing, Communications and Applications, 2024, 20(3): 76.
    [79] Yang WY, Zhou XY, Chen ZK, et al. AVoiD-DF: Audio-visual joint learning for detecting DeepFake. IEEE Transactions on Information Forensics and Security, 2023, 18: 2015–2029.
    [80] Cozzolino D, Pianese A, Nießner M, et al. Audio-visual person-of-interest DeepFake detection. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Vancouver: IEEE, 2023. 943–952.
    [81] Rössler A, Cozzolino D, Verdoliva L, et al. FaceForensics++: Learning to detect manipulated facial images. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2020. 1–11.
    [82] Li YZ, Yang X, Sun P, et al. Celeb-DF: A large-scale challenging dataset for DeepFake forensics. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 3204–3213.
    [83] Dolhansky B, Howes R, Pflaum B, et al. The DeepFake detection challenge (DFDC) preview dataset. arXiv:1910.08854, 2019.
    [84] Zhou TF, Wang WG, Liang ZY, et al. Face forensics in the wild. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021. 5774–5784.
    [85] Huang JJ, Wang XY, Du B, et al. DeepFake MNIST+: A DeepFake facial animation dataset. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops. Montreal: IEEE, 2021. 1973–1982.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

赵娅,郜明超,姚文达,徐锋.基于深度学习的伪造人脸检测技术综述.计算机系统应用,2025,34(4):1-17

复制
分享
文章指标
  • 点击次数:875
  • 下载次数: 352
  • HTML阅读次数: 78
  • 引用次数: 0
历史
  • 收稿日期:2024-10-11
  • 最后修改日期:2024-11-12
  • 在线发布日期: 2025-02-26
文章二维码
您是第11482545位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号