融合BERT和自注意力机制的张量图卷积网络文本分类
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

陕西省重点研发计划 (2024GX-YBXM-548)


Text Classification with Tensor Graph Convolutional Network Fusing BERT and Self-attention Mechanism
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    TensorGCN模型是图神经网络应用在文本分类领域的SOTA模型之一. 然而在处理文本语义信息方面, 该模型使用的LSTM难以完全地提取短文本语义特征, 且对复杂的语义处理效果不佳; 同时, 由于长文本中包含的语义及句法特征较多, 在进行图间异构信息共享时特征共享不完全, 影响文本分类的准确性. 针对这两个问题, 对TensorGCN模型进行改进, 提出融合BERT和自注意力机制的张量图卷积网络 (BTSGCN)文本分类方法. 首先, 使用BERT代替TensorGCN架构中的LSTM模块进行语义特征提取, 通过考虑给定单词两侧的周围单词来捕获单词之间的依赖关系, 更准确地提取短文本语义特征; 然后, 在图间传播时加入自注意力机制, 帮助模型更好地捕捉不同图之间的特征, 完成特征融合. 在MR、R8、R52和20NG这4个数据集上的实验结果表明BTSGCN相比于其他对比方法的分类准确度更高.

    Abstract:

    TensorGCN model is one of the state-of-the-art (SOTA) models applied by graph neural networks in the field of text classification. However, in terms of processing text semantic information, the long short-term memory (LSTM) used by the model has difficulty in completely extracting the semantic features of short text and performs poorly in handling complex semantic information. At the same time, due to the large number of semantic and syntactic features contained in long texts, feature sharing is incomplete when heterogeneous information is shared among graphs, which affects the accuracy of text classification. To solve these two problems, the TensorGCN model is improved, and a text classification method based on the tensor graph convolutional network fusing BERT and the self-attention mechanism (BTSGCN) is proposed. Firstly, BERT is used to replace the LSTM module in the TensorGCN architecture for semantic feature extraction. It captures the dependencies between words by considering the surrounding words on both sides of a given word, thus extracting the semantic features of short texts more accurately. Then, the self-attention mechanism is added during the propagation among graphs to help the model better capture the features among different graphs and complete the feature fusion. Experimental results on MR, R8, R52, and 20NG datasets show that BTSGCN has higher classification accuracy than other comparison methods.

    参考文献
    相似文献
    引证文献
引用本文

史文艺,朱欣娟.融合BERT和自注意力机制的张量图卷积网络文本分类.计算机系统应用,,():1-9

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-09-06
  • 最后修改日期:2024-10-10
  • 录用日期:
  • 在线发布日期: 2025-01-21
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号