基于LoRA的双阶段扩散模型水印方案
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

北京市教委科技一般项目(KM202410015001); 北京印刷学院校级项目(Ea202302, 27170123033, Ea202301)


Two-stage Diffusion Model Watermarking Scheme Based on LoRA
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    扩散模型的发展使得高质量图像生成变得更加便捷, 但同时引发了生成图像的版权保护问题. 现有研究通常在扩散过程中隐秘性地嵌入水印, 以提升水印鲁棒性. 然而, 目前现有基于扩散过程的水印方案集中于嵌入固定水印, 无法满足用户对水印多样化的需求. 此外, 还存在被恶意用户更换解码器规避水印的风险. 为了解决上述问题, 本文提出了基于LoRA的双阶段扩散模型水印方案. 首先, 该方案在水印编解码预训练阶段训练出水印编解码器, 保证水印嵌入的稳定性; 然后, 在U-Net微调阶段通过LoRA和自适应注意力机制, 使U-Net在保持生成质量的同时学习到第1阶段的水印模式, 实现多用户定制化. 实验表明, 该方案在图像一致性和水印鲁棒性上均优于现有方法. 在图像攻击下, 水印图像的FID距离提高了0.61%, 平均提取精度提升了4.9%.

    Abstract:

    The development of diffusion models has significantly enhanced the generation of high-quality images. However, it has also introduced notable challenges in the copyright protection of generated images. Existing research typically embeds watermarks invisibly during the diffusion process to increase their robustness. However, current watermarking methods based on diffusion processes focus on embedding a fixed watermark, failing to meet the diverse needs of users. In addition, malicious users may bypass watermarking by replacing the decoder, which presents a potential risk. To address these challenges, a two-stage watermarking scheme for diffusion models based on LoRA is proposed. In the first stage, the watermark codec is trained during pre-training to ensure stable watermark embedding. In the subsequent U-Net fine-tuning stage, LoRA and an adaptive attention mechanism are incorporated. This enables U-Net to learn watermark patterns from the first stage while maintaining image quality and supporting multi-user customization. Experimental results demonstrate that the proposed scheme outperforms existing methods in terms of image consistency and watermark robustness. Under image attacks, the FID distance of the watermarked images increases by only 0.61%, while the average extraction accuracy improves by 4.9%.

    参考文献
    相似文献
    引证文献
引用本文

白少杰,林立霞,胡子寒,袁艺林,曹鹏.基于LoRA的双阶段扩散模型水印方案.计算机系统应用,,():1-8

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-11-01
  • 最后修改日期:2024-11-29
  • 录用日期:
  • 在线发布日期: 2025-04-28
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号