AIPUB归智期刊联盟

微信公众号

网站二维码

2025年3月17日 1:22 星期一
首页 > 过刊浏览>2020年第29卷第7期 >48-55. DOI:10.15888/j.cnki.csa.007525
PDF HTML阅读 XML下载 导出引用 引用提醒
基于BERT-BiLSTM-CRF模型的中文实体识别
作者:
基金项目:

安徽省自然科学基金(1908085MF202); 国防科技大学校基金(ZK18-03-14)


Chinese Entity Recognition Based on BERT-BiLSTM-CRF Model
Author:
  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [28]
    • |
  • 相似文献 [20]
    • |
  • 引证文献
    • | |
  • 文章评论
    摘要:

    命名实体识别是自然语言处理的一项关键技术. 基于深度学习的方法已被广泛应用到中文实体识别研究中. 大多数深度学习模型的预处理主要注重词和字符的特征抽取, 却忽略词上下文的语义信息, 使其无法表征一词多义, 因而实体识别性能有待进一步提高. 为解决该问题, 本文提出了一种基于BERT-BiLSTM-CRF模型的研究方法. 首先通过BERT模型预处理生成基于上下文信息的词向量, 其次将训练出来的词向量输入BiLSTM-CRF模型做进一步训练处理. 实验结果表明, 该模型在MSRA语料和人民日报语料库上都达到相当不错的结果, F1值分别为94.65%和95.67%.

    Abstract:

    Named Entity Recognition is a key technology in natural language processing, and the methods based on deep learning have been widely used in Chinese entity recognition. Most deep learning models focus on the feature extraction of words and characters, but ignore the semantic information of word context, therefore, they cannot represent polysemy, and the performance of entity recognition needs to be further improved. In order to solve this problem, this study proposes a method based on the BERT-BiLSTM-CRF model. First, word vectors based on context information are generated by the pretreatment of BERT model, and then the trained word vector is input into BiLSTM-CRF model for further training. The experimental result shows that the proposed model achieves sound results and reaches F1-score of 94.65% and 95.67% respectively in the MSRA corpus and People’s Daily.

    参考文献
    [1] 彭春艳, 张晖, 包玲玉, 等. 基于条件随机域的生物命名实体识别. 计算机工程, 2009, 35(22): 197-199. [doi: 10.3969/j.issn.1000-3428.2009.22.067
    [2] 鞠久朋, 张伟伟, 宁建军, 等. CRF与规则相结合的地理空间命名实体识别. 计算机工程, 2011, 37(7): 210-212, 215. [doi: 10.3969/j.issn.1000-3428.2011.07.071
    [3] 乐娟, 赵玺. 基于HMM的京剧机构命名实体识别算法. 计算机工程, 2013, 39(6): 266-271. [doi: 10.3969/j.issn.1000-3428.2013.06.059
    [4] Hammerton J. Named entity recognition with long short-term memory. Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL. Stroudsburg, PA, USA. 2003. 172-175.
    [5] Lample G, Ballesteros M, Subramanian S, et al. Neural architectures for named entity recognition. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, CA, USA. 2016. 260-270.
    [6] Pinheiro PHO, Collobert R. Recurrent convolutional neural networks for scene parsing. Proceedings of ICML. Beijing, China. 2014. 82-90.
    [7] Huang ZH, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv: 1508.01991, 2015.
    [8] Chiu JPC, Nichols E. Named entity recognition with bidirectional LSTM-CNNs. Transactions of the Association for Computational Linguistics, 2016, 4: 357-370. [doi: 10.1162/tacl_a_00104
    [9] 李丽双, 郭元凯. 基于CNN-BLSTM-CRF模型的生物医学命名实体识别. 中文信息学报, 2018, 32(1): 116-122. [doi: 10.3969/j.issn.1003-0077.2018.01.015
    [10] Luo L, Yang ZH, Yang P, et al. An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition. Bioinformatics, 2018, 34(8): 1381-1388. [doi: 10.1093/bioinformatics/btx761
    [11] Wu FZ, Liu JX, Wu CH, et al. Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation. The World Wide Web Conference. New York, NY, USA. 2019. 3342-3348.
    [12] 秦娅, 申国伟, 赵文波, 等. 基于深度神经网络的网络安全实体识别方法. 南京大学学报(自然科学), 2019, 55(1): 29-40
    [13] 武惠, 吕立, 于碧辉. 基于迁移学习和BiLSTM-CRF的中文命名实体识别. 小型微型计算机系统, 2019, 40(6): 1142-1147
    [14] 王红斌, 沈强, 线岩团. 融合迁移学习的中文命名实体识别. 小型微型计算机系统, 2017, 38(2): 346-351
    [15] 王银瑞, 彭敦陆, 陈章, 等. Trans-NER: 一种迁移学习支持下的中文命名实体识别模型. 小型微型计算机系统, 2019, 40(8): 1622-1626. [doi: 10.3969/j.issn.1000-1220.2019.08.008
    [16] Dong CH, Zhang JJ, Zong CQ, et al. Character-based LSTM-CRF with radical-level features for Chinese named entity recognition. In: Lin CY, Xue N, Zhao D, et al., eds. Natural Language Understanding and Intelligent Applications. Cham: Springer, 2016.239-250.
    [17] 刘晓俊, 辜丽川, 史先章. 基于Bi-LSTM和注意力机制的命名实体识别. 洛阳理工学院学报(自然科学版), 2019, 29(1): 65-70, 77
    [18] Zhang Y, Yang J. Chinese NER using lattice LSTM. arXiv preprint arXiv: 1805.02023, 2018.
    [19] Liu W, Xu TG, Xu QH, et al. An encoding strategy based word-character LSTM for Chinese NER. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, MN, USA. 2019. 2379-2389.
    [20] 王蕾, 谢云, 周俊生, 等. 基于神经网络的片段级中文命名实体识别. 中文信息学报, 2018, 32(3): 84-90, 100. [doi: 10.3969/j.issn.1003-0077.2018.03.012
    [21] Devlin J, Chang MW, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 1810.04805, 2018.
    [22] Souza F, Nogueira R, Lotufo R. Portuguese named entity recognition using BERT-CRF. arXiv preprint arXiv: 1909.10649, 2019.
    [23] Straková J, Straka M, Hajič J. Neural architectures for nested NER through linearization. arXiv preprint arXiv: 1908.06926, 2019.
    [24] Jawahar G, Sagot B, Seddah D. What does BERT learn about the structure of language? Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy. 2019. 3651-3657.
    [25] Cui YM, Che WX, Liu T, et al. Pre-training with whole word masking for Chinese BERT. arXiv preprint arXiv: 1906.08101, 2019.
    [26] Graves A, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Network, 2005, 18(5-6): 602-610
    [27] Jia YZ, Xu XB. Chinese named entity recognition based on CNN-BiLSTM-CRF. 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS). Beijing, China. 2018. 1-4.
    [28] 李妮, 关焕梅, 杨飘, 等. 基于BERT-IDCNN-CRF的中文命名实体识别方法. 山东大学学报(理学版), 2020, 55(1): 102-109
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

谢腾,杨俊安,刘辉.基于BERT-BiLSTM-CRF模型的中文实体识别.计算机系统应用,2020,29(7):48-55

复制
分享
文章指标
  • 点击次数:2709
  • 下载次数: 9853
  • HTML阅读次数: 20643
  • 引用次数: 0
历史
  • 收稿日期:2019-12-22
  • 最后修改日期:2020-01-19
  • 在线发布日期: 2020-07-04
  • 出版日期: 2020-07-15
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会软件学报
您是第10787628位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码：100190
电话：010-62661041 传真： Email：csa (a) iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号