基于图卷积神经网络的自注意力的融合节点分类框架
作者:
基金项目:

智能农业广州重点实验室 (201902010081); 广东省科技规划项目 (2017A040406023); 广州市科技规划项目 (201804010353)


Fusion Node Classification Framework Based on Self-attention in Graph Convolutional Networks
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [20]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    图神经网络因其强大的建模能力引起广泛关注, 常常被用来解决图上的节点分类任务. 现阶段常用的以图卷积神经网络 (graph convolutional network, GCN)为内核的模型解决此类问题, 但往往因为出现过拟合与过平滑而导致深层的节点嵌入表示效果并不好. 因此, 本文提出了一种基于GCN内核的结合残差连接与自注意力方法——GCNRN模型, 以提升GCN的泛化能力. 同时, 为了整合更深入的信息, 本文引入融合机制, 采用模糊积分融合多个分类器, 最终提高模型测试精度. 为了验证所提出方法的优越性, 本文采用ogbn-arxiv与常用的引文数据集进行了对比实验. GCNRN模型与多个以GCN为内核的现有模型相比, 节点分类准确率平均提高了2%, 且避免了传统的过拟合和过平滑现象. 此外, 实验结果表明, 增加了基于模糊积分的融合模块的多分类器模型比传统融合方法具有更好的分类效果.

    Abstract:

    Graph neural networks (GNNs) have attracted widespread attention due to their powerful modeling capabilities, and they are often used to solve node classification tasks on graphs. At this stage, the commonly used model with the graph convolutional network (GCN) as the core solves such problems. However, due to over-fitting and over-smoothing, the deep node embedding representation effect is not positive. Therefore, this study proposes a graph convolutional neural residual networks (GCNRN) model that combines residual connection and self-attention based on GCN kernel to improve the generalization ability of GCN. At the same time, in order to integrate more in-depth information, this study introduces a fusion mechanism, uses fuzzy integral to fuse multiple classifiers, and finally improves the model testing accuracy. In order to verify the superiority of the proposed method, this study uses ogbn-arxiv and commonly used citation datasets to conduct comparative experiments. Compared with many existing models with GCN as the core, the GCNRN model has an average improvement of node classification accuracy by 2% and avoids the traditional over-fitting and over-smoothing phenomena. In addition, the experimental results show that the multi-classifier model with the fusion module based on fuzzy integral has a better classification effect than the traditional fusion method.

    参考文献
    [1] Kipf TN, Welling M. Semi-supervised classification with graph convolutional networks. arXiv:1609.02907, 2016.
    [2] Veličković P, Cucurull G, Casanova A, et al. Graph attention networks. arXiv:1710.10903, 2017.
    [3] Hamilton WL, Ying R, Leskovec J. Inductive representation learning on large graphs. Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 1025–1035.
    [4] Xu KL, Li CT, Tian YL, et al. Representation learning on graphs with jumping knowledge networks. Proceedings of the 35th International Conference on Machine Learning. Stockholmsmässan: PMLR, 2018. 5453–5462.
    [5] Chen M, Wei Z, Huang Z, et al. Simple and deep graph convolutional networks. Proceedings of the 2020 International Conference on Machine Learning. PMLR, 2020. 1725–1735.
    [6] Li G, Xiong C, Thabet A, et al. DeeperGCN: All you need to train deeper GCNS. arXiv:2006.07739, 2020.
    [7] 胡毅, 瞿博阳, 梁静, 等. 进化集成学习算法综述. 智能科学与技术学报, 2021, 3(1): 18–33. [doi: 10.11959/j.issn.2096-6652.02103
    [8] Wang WY, Tran D, Feiszli M. What makes training multi-modal classification networks hard. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 12692–12702.
    [9] Izmailov P, Podoprikhin D, Garipov T, et al. Averaging weights leads to wider optima and better generalization. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence. Monterey: AUAI Press, 2018. 876–885.
    [10] Micheli A. Neural network for graphs: A contextual constructive approach. IEEE Transactions on Neural Networks, 2009, 20(3): 498–511. [doi: 10.1109/TNN.2008.2010350
    [11] Bruna J, Zaremba W, Szlam A, et al. Spectral networks and locally connected networks on graphs. arXiv:1312.6203, 2013.
    [12] Lou SH, Feng YX, Li ZW, et al. Two-additive fuzzy measure-based information integration approach to product design alternative evaluation. Journal of Industrial Information Integration, 2022, 25: 100247. [doi: 10.1016/j.jii.2021.100247
    [13] Wu YX, He KM. Group normalization. arXiv:1803.08494. 2018. 3–19.
    [14] Shen ZW, Yang HZ, Zhang SJ. Optimal approximation rate of ReLU networks in terms of width and depth. Journal de Mathématiques Pures et Appliquées, 2022, 157: 101–135
    [15] Kong XT, Liu XN, Gu JJ, et al. Reflash dropout in image super-resolution. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022. 5992–6002.
    [16] Grabisch M, Labreuche C. Bi-capacities—II: The Choquet integral. Fuzzy Sets and Systems, 2005, 151(2): 237–259. [doi: 10.1016/j.fss.2004.08.013
    [17] 王晓君. 基于模糊积分的多分类器融合研究[硕士学位论文]. 保定: 河北大学, 2005.
    [18] 王业琳, 宁新宝, 尹义龙. 指纹图像细化算法的研究. 南京大学学报(自然科学), 2003, 39(4): 468–475
    [19] Hu W, Fey M, Zitnik M, et al. Open graph benchmark: Datasets for machine learning on graphs. Advances in Neural Information Processing Systems, 2020, 33: 22118–22133.
    [20] Rumelhart DE, Hinton GE, Williams RJ. Learning internal representations by error propagation. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations. Cambridge: MIT Press, 1987.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

姜发健,王金凤,招奕钧,郑志燊.基于图卷积神经网络的自注意力的融合节点分类框架.计算机系统应用,2023,32(7):251-260

复制
分享
文章指标
  • 点击次数:612
  • 下载次数: 2151
  • HTML阅读次数: 3362
  • 引用次数: 0
历史
  • 收稿日期:2023-01-02
  • 最后修改日期:2023-01-19
  • 在线发布日期: 2023-05-24
文章二维码
您是第11206990位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号