融合TuckER嵌入和强化学习的知识推理
作者:
基金项目:

国家自然科学基金(61806072); 天津市自然科学基金(19JCZDJC40000); 河北省高等学校科学技术研究项目(QN2021213); 河北省自然科学基金(F2020202008)


Knowledge Reasoning Combining TuckER Embedding and Reinforcement Learning
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [29]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    知识推理是补全知识图谱的重要方法, 旨在根据图谱中已有的知识, 推断出未知的事实或关系. 针对多数推理方法仍存在没有充分考虑实体对之间的路径信息, 且推理效率偏低、可解释性差的问题, 提出了将TuckER嵌入和强化学习相结合的知识推理方法TuckRL (TuckER embedding with reinforcement learning). 首先, 通过TuckER嵌入将实体和关系映射到低维向量空间, 在知识图谱环境中采用策略引导的强化学习算法对路径推理过程进行建模, 然后在路径游走进行动作选择时引入动作修剪机制减少无效动作的干扰, 并将LSTM作为记忆组件保存智能体历史动作轨迹, 促使智能体更准确地选择有效动作, 通过与知识图谱的交互完成知识推理. 在3个主流大规模数据集上进行了实验, 结果表明TuckRL优于现有的大多数推理方法, 说明将嵌入和强化学习相结合的方法用于知识推理的有效性.

    Abstract:

    Knowledge reasoning is an important method to complement a knowledge graph, which aims to infer unknown facts or relations according to the existing knowledge in the graph. As the path information between entity pairs is not fully considered in most reasoning methods, the reasoning shows low efficiency and poor interpretability. To solve this problem, this study proposes TuckER embedding with reinforcement learning (TuckRL), a knowledge reasoning method that combines TuckER embedding and reinforcement learning (RL). First, entities and relations are mapped to low-dimensional vector space through TuckER embedding, and the path reasoning process is modeled using RL guided by strategies in the knowledge graph environment. Then, the action pruning mechanism is introduced to reduce the interference of invalid actions for action selection during path walking, and LSTM is used as the memory component to preserve the agent’s historical action trajectory. In this way, the agent can more accurately select valid actions and can complete knowledge reasoning by interaction with the knowledge graph. The experiments on three mainstream large-scale datasets indicate that TuckRL is superior to most of the existing methods, which demonstrates the effectiveness of combining embedding and RL for knowledge reasoning.

    参考文献
    [1] 黄恒琪, 于娟, 廖晓, 等. 知识图谱研究综述. 计算机系统应用, 2019, 28(6): 1–12. [doi: 10.15888/j.cnki.csa.006915
    [2] Miller GA. WordNet: A lexical database for English. Communications of the ACM, 1995, 38(11): 39–41. [doi: 10.1145/219717.219748
    [3] Carlson A, Betteridge J, Kisiel B, et al. Toward an architecture for never-ending language learning. Proceedings of the 24th AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2010. 1306–1313.
    [4] Bollacker KD, Evans C, Paritosh P, et al. Freebase: A collaboratively created graph database for structuring human knowledge. Vancouver: ACM, 2008. 1247–1250.
    [5] 封皓君, 段立, 张碧莹. 面向知识图谱的知识推理综述. 计算机系统应用, 2021, 30(10): 21–30. [doi: 10.15888/j.cnki.csa.008137
    [6] 官赛萍, 靳小龙, 贾岩涛, 等. 面向知识图谱的知识推理研究进展. 软件学报, 2018, 29(10): 2966–2994. [doi: 10.13328/j.cnki.jos.005551
    [7] Yang BS, Yih WT, He XD, et al. Embedding entities and relations for learning and inference in knowledge bases. arXiv: 1412.6575, 2014.
    [8] Bordes A, Usunier N, Garcia-Durán A, et al. Translating embeddings for modeling multi-relational data. Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe: Curran Associates Inc., 2013. 2787–2795.
    [9] Dettmers T, Minervini P, Stenetorp P, et al. Convolutional 2D knowledge graph embeddings. Proceedings of the 32nd AAAI Conference on Artificial Intelligence. New Orleans: AAAI, 2018. 1811–1818.
    [10] Lao N, Mitchell TM, Cohen WW. Random walk inference and learning in a large scale knowledge base. Proceedings of the Conference on Empirical Methods in Natural Language Processing. Edinburgh: ACM, 2011. 529–539.
    [11] Bellman R. A Markovian decision process. Journal of Mathematics and Mechanics, 1957, 6(5): 679–684
    [12] Williams RJ. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 1992, 8(3–4): 229–256. [doi: 10.1007/BF00992696
    [13] 李茹杨, 彭慧民, 李仁刚, 等. 强化学习算法与应用综述. 计算机系统应用, 2020, 29(12): 13–25. [doi: 10.15888/j.cnki.csa.007701
    [14] Xiong WH, Hoang T, Wang WY. DeepPath: A reinforcement learning method for knowledge graph reasoning. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen: Association for Computational Linguistics, 2017. 564–573.
    [15] Wang Z, Zhang JW, Feng JL, et al. Knowledge graph embedding by translating on hyperplanes. Proceedings of the 28th AAAI Conference on Artificial Intelligence. Quebec City: AAAI, 2014. 1112–1119.
    [16] Ji GL, He SZ, Xu LH, et al. Knowledge graph embedding via dynamic mapping matrix. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Beijing: Association for Computational Linguistics, 2015. 687–696.
    [17] 陈文杰, 文奕, 张鑫, 等. 一种改进的基于TransE知识图谱表示方法. 计算机工程, 2020, 46(5): 63–69, 77
    [18] Trouillon T, Welbl J, Riedel S, et al. Complex embeddings for simple link prediction. Proceedings of the 33nd International Conference on Machine Learning. New York: PMLR, 2016. 2071–2080.
    [19] Balažević I, Allen C, Hospedales TM. TuckER: Tensor factorization for knowledge graph completion. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong: Association for Computational Linguistics, 2019. 5185–5194.
    [20] Lin YK, Liu ZY, Luan HB, et al. Modeling relation paths for representation learning of knowledge bases. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon: Association for Computational Linguistics, 2015. 705–714.
    [21] Das R, Neelakantan A, Belanger D, et al. Chains of reasoning over entities, relations, and text using recurrent neural networks. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Valencia: Association for Computational Linguistics, 2017. 132–141.
    [22] Wang X, Wang DX, Xu CR, et al. Explainable reasoning over knowledge graphs for recommendation. Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Honolulu: AAAI, 2019. 5329–5336.
    [23] Das R, Dhuliawala S, Zaheer M, et al. Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. 6th International Conference on Learning Representations. Vancouver: OpenReview.net, 2018.
    [24] Shen YL, Chen JS, Huang PS, et al. M-Walk: Learning to walk over graphs using Monte Carlo tree search. Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montreal: Curran Associates Inc., 2018. 6787–6798.
    [25] Li RP, Cheng X. DIVINE: A generative adversarial imitation learning framework for knowledge graph reasoning. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong: Association for Computational Linguistics, 2019. 2642–2651.
    [26] Meilicke C, Chekol MW, Fink M, et al. Reinforced anytime bottom up rule learning for knowledge graph completion. arXiv: 2004.04412, 2000.
    [27] Lei DR, Jiang GR, Gu XT, et al. Learning collaborative agents with rule guidance for knowledge graph reasoning. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2020. 8541–8547.
    [28] 崔员宁, 李静, 陈琰, 等. TransPath: 一种基于深度迁移强化学习的知识推理方法. 小型微型计算机系统, 2022, 43(3): 536–543
    [29] Nguyen DQ, Nguyen TD, Nguyen DQ, et al. A novel embedding model for knowledge base completion based on convolutional neural network. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). New Orleans: Association for Computational Linguistics, 2018. 327–333.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

于铁忠,罗婧,王利琴,董永峰.融合TuckER嵌入和强化学习的知识推理.计算机系统应用,2022,31(9):127-135

复制
分享
文章指标
  • 点击次数:851
  • 下载次数: 2234
  • HTML阅读次数: 1950
  • 引用次数: 0
历史
  • 收稿日期:2021-12-16
  • 最后修改日期:2022-01-18
  • 在线发布日期: 2022-06-28
文章二维码
您是第11278034位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号