嵌入和梯度双向压缩的高效纵向联邦学习
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

国家重点研发计划 (2022YFC3303500)


Efficient Vertical Federated Learning Based on Embedding and Gradient Bidirectional Compression
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    纵向联邦学习在不泄露数据隐私的前提下, 通过联合多方本地数据特征, 共同训练目标模型, 提高数据利用价值, 受到业界公司和机构的广泛关注. 在训练过程中, 客户端上传的中间嵌入及服务器返回的梯度信息需要巨大的通信量, 通信成本成为限制其实际应用的关键瓶颈. 如何通过有效的算法设计减少通信量、提高通信效率成为当前研究的热点之一. 本文针对纵向联邦学习通信效率问题, 提出基于嵌入和梯度双向压缩的高效压缩算法, 对客户端上传的嵌入表示, 采用改进的稀疏化方法并结合缓存重用机制, 对服务器分发的梯度信息, 采用离散量化与哈夫曼编码结合的机制. 实验结果表明, 本文算法能够在准确率与无压缩场景保持相当的前提下, 降低约85%的通信量, 提高通信效率, 减少整体训练时间.

    Abstract:

    Vertical federated learning improves the value of data utilization by combining local data features from multiple parties and jointly training the target model without leaking data privacy. It has received widespread attention from companies and institutions in the industry. During the training process, the intermediate embeddings uploaded by clients and the gradients returned by the server require a huge amount of communication, and thus the communication cost becomes a key bottleneck limiting the practical application of vertical federated learning. Consequently, current research focuses on designing effective algorithms to reduce the communication amount and improve communication efficiency. To improve the communication efficiency of vertical federated learning, this study proposes an efficient compression algorithm based on embedding and gradient bidirectional compression. For the embedding representation uploaded by the client, an improved sparsification method combined with a cache reuse mechanism is employed. For the gradient information distributed by the server, a mechanism combining discrete quantization and Huffman coding is used. Experimental results show that the proposed algorithm can reduce the communication volume by about 85%, improve communication efficiency, and reduce the overall training time while maintaining almost the same accuracy as the uncompressed scenario.

    参考文献
    相似文献
    引证文献
引用本文

张宇航,嵩天.嵌入和梯度双向压缩的高效纵向联邦学习.计算机系统应用,2024,33(10):190-197

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-04-01
  • 最后修改日期:2024-04-29
  • 录用日期:
  • 在线发布日期: 2024-08-21
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号