基于改进CycleGAN的人脸剪纸自动生成
作者:
基金项目:

国家自然科学基金(61772172)


Automatic Portrait Paper-cut Generation Based on Improved CycleGAN
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [29]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    针对人脸剪纸手工设计难度大, 制作周期长等问题, 本文首次利用生成对抗网络生成高质量人脸剪纸. 面向人脸剪纸艺术特点, 提出了一种基于CycleGAN的改进网络: 1) 在原始CycleGAN生成器中引入CBAM注意力模块, 增强网络特征提取能力; 2) 引入针对鼻、眼、唇等关键面部区域的局部鉴别器, 提升人脸剪纸中以上区域的生成效果; 3)设计基于图像边缘信息与SSIM的损失函数, 取代CycleGAN的前向循环一致损失, 消除所得人脸剪纸中的阴影. 相较于其他人脸剪纸自动生成方法, 本文方法可快速生成与原始人脸相似度高、线条连续流畅、具有艺术美感的人脸剪纸. 此外, 本文还提出了一种人脸剪纸连通性后处理方法, 使所得结果更符合中国传统剪纸整体连通的特点.

    Abstract:

    Aiming at the difficulty in manually designing portrait paper-cuts, this study employs the generative adversarial network (GAN) to generate high-quality portrait paper-cuts for the first time. Based on the artistic characteristics of portrait paper-cuts, an improved network based on CycleGAN is proposed. 1) The CBAM attention module is introduced into the CycleGAN generator to enhance the feature extraction of the network. 2) The local discriminator for key facial regions such as nose, eyes, and lips is introduced to improve the generation effect of the above areas in generated portrait paper-cuts. 3) A new loss function is designed based on image edge information and SSIM, which will be adopted to replace the original forward cycle-consistency loss of CycleGAN and eliminate the shadows in the portrait paper-cuts. Compared with other automatic generation methods of portrait paper-cuts, the proposed method can quickly generate paper-cuts featuring high similarity to the original human face, continuous and smooth lines, and aesthetic beauty. Additionally, this study also puts forward a post-processing method of portrait paper-cut connectivity to make the obtained results more consistent with the overall connectivity of traditional Chinese paper-cuts.

    参考文献
    [1] 王晓宇, 赵妍, 刘泽云, 等. 基于五官特征的人脸剪纸生成方法研究. 计算机工程与应用, 2018, 54(17): 208–213. [doi: 10.3778/j.issn.1002-8331.1704-0334
    [2] Lei HY, Qiu WL. The conversion from line drawings to paintings with paper-cutting designs. Proceedings of the 2011 International Conference on Computer Science and Network Technology. Harbin: IEEE, 2011. 2783–2786.
    [3] Meng M, Zhao MT, Zhu SC. Artistic paper-cut of human portraits. Proceedings of the 18th ACM International Conference on Multimedia. Firenze: ACM, 2010. 931–934.
    [4] Dang K, Song GL, Ming AL. Chinese paper-cut generated from human portrait. Proceedings of the 2016 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW). Nantou: IEEE, 2016. 1–2.
    [5] Xu J, Kaplan CS, Mi XF. Computer-generated papercutting. Proceedings of the 15th Pacific Conference on Computer Graphics and Applications (PG 2007). Maui: IEEE, 2007. 343–350.
    [6] Xu J, Kaplan CS. Artistic thresholding. Proceedings of the 6th International Symposium on Non-photorealistic Animation and Rendering. Annecy: ACM, 2008. 39–47.
    [7] Wang TH, Collomosse JP, Hunter A, et al. Learnable stroke models for example-based portrait painting. Proceedings of the 2013 British Machine Vision Conference. Bristol: BMVA Press, 2013.
    [8] Berger I, Shamir A, Mahler M, et al. Style and abstraction in portrait sketching. ACM Transactions on Graphics, 2013, 32(4): 1–12. [doi: 10.1145/2461912.2461964
    [9] Zhao MT, Zhu SC. Portrait painting using active templates. Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Non-photorealistic Animation and Rendering. Vancouver: ACM, 2011. 117–124.
    [10] Zhang Y, Dong WM, Ma CY, et al. Data-driven synthesis of cartoon faces using different styles. IEEE Transactions on Image Processing, 2017, 26(1): 464–478. [doi: 10.1109/TIP.2016.2628581
    [11] Chen H, Liu ZQ, Rose C, et al. Example-based composite sketching of human portraits. Proceedings of the 3rd International Symposium on Non-photorealistic Animation and Rendering. Annecy: ACM, 2004. 95–153.
    [12] Rosin PL, Lai YK. Non-photorealistic rendering of portraits. Proceedings of the 2015 Workshop on Computational Aesthetics. Istanbul: Eurographics Association, 2015. 159–170.
    [13] Fišer J, Jamriška O, Simons D, et al. Example-based synthesis of stylized facial animations. ACM Transactions on Graphics, 2017, 36(4): 1–11. [doi: 10.1145/3072959.3073660
    [14] Selim A, Elgharib M, Doyle L. Painting style transfer for head portraits using convolutional neural networks. ACM Transactions on Graphics, 2016, 35(4): 129. [doi: 10.1145/2897824.2925968
    [15] Gatys LA, Ecker AS, Bethge M. Image style transfer using convolutional neural networks. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016. 2414–2423.
    [16] Kaur P, Zhang H, Dana K. Photo-realistic facial texture transfer. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa: IEEE, 2019. 2097–2105.
    [17] Zhang YB, Li MH, Li RH, et al. Exact feature distribution matching for arbitrary style transfer and domain generalization. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022. 8025–8035.
    [18] Yang S, Jiang LM, Liu ZW, et al. Pastiche master: Exemplar-based high-resolution portrait style transfer. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022. 7683–7692.
    [19] Isola P, Zhu JY, Zhou TH, et al. Image-to-image translation with conditional adversarial networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017. 5967–5976.
    [20] Zhu JY, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017. 2242–2251.
    [21] Shi YC, Deb D, Jain AK. WarpGAN: Automatic caricature generation. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 10754–10763.
    [22] Choi Y, Uh Y, Yoo J, et al. StarGANv2: Diverse image synthesis for multiple domains. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 8185–8194.
    [23] Woo S, Park J, Lee JY, et al. CBAM: Convolutional block attention module. Proceedings of the 15th European Conference on Computer Vision. Munich: Springer, 2018. 3–19.
    [24] Wang Z, Bovik AC, Sheikh HR, et al. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. [doi: 10.1109/TIP.2003.819861
    [25] Luo L, Xue DY, Feng XL. Ehanet: An effective hierarchical aggregation network for face parsing. Applied Sciences, 2020, 10(9): 3135. [doi: 10.3390/app10093135
    [26] Xie SN, Tu ZW. Holistically-nested edge detection. Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015. 1395–1403.
    [27] Qin XB, Zhang ZC, Huang CY, et al. U2-Net: Going deeper with nested U-structure for salient object detection. Pattern Recognition, 2020, 106: 107404. [doi: 10.1016/j.patcog.2020.107404
    [28] Shrivastava A, Pfister T, Tuzel O, et al. Learning from simulated and unsupervised images through adversarial training. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017. 2242–2251.
    [29] Heusel M, Ramsauer H, Unterthiner T, et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 6629–6640.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

牟玮,童晶,韦剑,张明懿.基于改进CycleGAN的人脸剪纸自动生成.计算机系统应用,2023,32(12):1-11

复制
分享
文章指标
  • 点击次数:886
  • 下载次数: 1828
  • HTML阅读次数: 1021
  • 引用次数: 0
历史
  • 收稿日期:2023-05-17
  • 最后修改日期:2023-06-26
  • 在线发布日期: 2023-10-25
文章二维码
您是第11117171位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号