改进GL-GIN的多意图识别和槽填充联合模型
作者:
基金项目:

国家自然科学基金(62076103)


Multi-intent Detection and Slot Filling Joint Model of Improved GL-GIN
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [22]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    在当前自然语言处理多意图识别模型研究中, 存在建模方式均为从意图到插槽的单一方向的信息流建模, 忽视了插槽到意图的信息流交互建模研究, 意图识别任务易于混淆且错误捕获其他意图信息, 上下文语义特征提取质量不佳, 有待进一步提升等问题. 本文以当前先进的典型代表GL-GIN模型为基础, 进行优化改进, 探索了插槽到意图的交互建模方法, 运用槽到意图的单向注意力层, 计算插槽到意图的注意力得分, 纳入注意力机制, 利用插槽到意图的注意力得分作为连接权重, 使其可以传播和聚集与意图相关的插槽信息, 使意图重点关注与其相关的插槽信息, 从而实现多意图识别模型的双向信息流动; 同时, 引入BERT模型作为编码层, 以提升了语义特征提取质量. 实验表明, 该交互建模方法效果提升明显, 与原GL-GIN模型相比, 在两个公共数据集(MixATIS和MixSNIPS)上, 新模型的总准确率分别提高了5.2%和9%.

    Abstract:

    In the current research on multi-intention recognition models of natural language processing, information flow is only modeled from intention to slot, and the research on the interactive modeling of information flow from slot to intention is ignored. In addition, the task of intention recognition is easy to be confused, and other intention information is wrongly captured. The quality of contextual semantic feature extraction is poor and needs to be improved. In order to solve these problems, this study optimizes the current advanced typical GL-GIN (global-locally graph interaction network) model, explores the interactive modeling method from slot to intention, and uses the one-way attention layer from slot to intention. Furthermore, the study calculates the attention score from slot to intention, incorporates the attention mechanism, and uses the attention score from slot to intention as the connection weight. As a result, it can propagate and gather intention-related slot information and make the intention focus on the slot information that is relevant to it, so as to realize the bidirectional information flow of the multi-intention recognition model. At the same time, the BERT model is introduced as the coding layer to improve the quality of semantic feature extraction. Experiments show that the effect of this interactive modeling method is significantly improved. Compared with that of the original GL-GIN model, the overall accuracy of the new model on two public datasets (MixATIS and MixSNIPS) is increased by 5.2% and 9%, respectively.

    参考文献
    [1] 刘娇, 李艳玲, 林民. 人机对话系统中意图识别方法综述. 计算机工程与应用, 2019, 55(12): 1–7, 43. [doi: 10.3778/j.issn.1002-8331.1902-0129
    [2] Goo CW, Gao G, Hsu YK, et al. Slot-gated modeling for joint slot filling and intent prediction. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans: ACL, 2018. 753–757.
    [3] Li CL, Li L, Qi J. A self-attentive model with gate mechanism for spoken language understanding. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018. 3824–3833.
    [4] Liu YJ, Meng FD, Zhang JC, et al. CM-Net: A novel collaborative memory network for spoken language understanding. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong: ACL, 2019. 1051–1060.
    [5] E HH, Niu PQ, Chen ZF, et al. A novel bi-directional interrelated model for joint intent detection and slot filling. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 5467–5471.
    [6] Zhang XD, Wang HF. A joint model of intent determination and slot filling for spoken language understanding. Proceedings of the 25th International Joint Conference on Artificial Intelligence. New York: AAAI Press, 2016. 2993–2999.
    [7] Qin LB, Che WX, Li YM, et al. A stack-propagation framework with token-level intent detection for spoken language understanding. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong: ACL, 2019. 2078–2087.
    [8] Hakkani-Tür D, Tür G, Celikyilmaz A, et al. Multi-domain joint semantic frame parsing using bi-directional RNN-LSTM. Proceedings of the 17th Annual Conference of the International Speech Communication Association (INTERSPEECH 2016). San Francisco, 2016. 715–719.
    [9] Xia CY, Zhang CW, Yan XH, et al. Zero-shot user intent detection via capsule neural networks. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018. 3090–3099.
    [10] Zhang CW, Li YL, Du N, et al. Joint slot filling and intent detection via capsule neural networks. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 5259–5267.
    [11] Wu D, Ding L, Lu F, et al. SlotRefine: A fast non-autoregressive model for joint intent detection and slot filling. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). ACL, 2020. 1932–1937.
    [12] Qin LB, Liu TL, Che WX, et al. A co-interactive transformer for joint slot filling and intent detection. Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2021). Toronto: IEEE, 2021. 8193–8197.
    [13] Qin LB, Wei FX, Xie TB, et al. GL-GIN: Fast and accurate non-autoregressive model for joint multiple intent detection and slot filling. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. ACL, 2021. 178–188.
    [14] Gangadharaiah R, Narayanaswamy B. Joint multiple intent detection and slot labeling for goal-oriented dialog. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis: ACL, 2019. 564–569.
    [15] Qin LB, Xu X, Che WX, et al. AGIF: An adaptive graph-interactive framework for joint multiple intent detection and slot filling. Proceedings of the 2020 Findings of the Association for Computational Linguistics: EMNLP 2020. ACL, 2020. 1807–1816.
    [16] Veličković P, Cucurull G, Casanova A, et al. Graph attention networks. arXiv:1710.10903, 2017.
    [17] Devlin J, Chang MW, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: ACL, 2019. 4171–4186.
    [18] Cao Y, Fang M, Tao DC. BAG: Bi-directional attention entity graph convolutional network for multi-hop reasoning question answering. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: ACL, 2019. 357–362.
    [19] Tian YH, Chen GM, Song Y, et al. Dependency-driven relation extraction with attentive graph convolutional networks. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. ACL, 2021. 4458–4471.
    [20] Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014.
    [21] Liu B, Lane IR. Attention-based recurrent neural network models for joint intent detection and slot filling. Proceedings of the 17th Annual Conference of the International Speech Communication Association. San Francisco, 2016. 685–689.
    [22] Wang Y, Shen YL, Jin HX. A bi-model based RNN semantic frame parsing model for intent detection and slot filling. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans: Association for Computational Linguistics, 2018. 309–314.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

邓飞燕,陈壹华,陈禧琳,李杰鸿.改进GL-GIN的多意图识别和槽填充联合模型.计算机系统应用,2023,32(7):75-83

复制
分享
文章指标
  • 点击次数:912
  • 下载次数: 2017
  • HTML阅读次数: 1456
  • 引用次数: 0
历史
  • 收稿日期:2022-12-22
  • 最后修改日期:2023-02-03
  • 在线发布日期: 2023-04-17
文章二维码
您是第11336308位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号