Text Similarity Matching Model Based on Positive and Negative Samples and Bi-LSTM
CSTR:
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [20]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    Similarity matching is crucial for natural language processing and also for extracting answers from the question answering system. This study proposes a model of text similarity matching based on positive and negative samples and Bi-LSTM. Firstly, this model constructs question answering pairs for positive and negative samples in model training, improving the similarity between a question and its correct answer. Secondly, it applies the dual-layer word vector embedding for pre-training to solve the experimental error caused by segmentation mistakes. Thirdly, it adopts the internal attention mechanism before feature extraction to solve the backward offset of the characteristic vectors caused by the attention mechanism. Then this model trains the data on the Bi-LSTM neural network to retain important temporal characteristics. Finally, it puts forward a similarity calculation function including semantic information to calculate similarity at the semantic level. The model proposed in this study is simulated on the public data set DuReader and compared with other models. The experimental results show that the proposed model has high accuracy and good robustness, and the accuracy of top-1 reaches 78.34%.

    Reference
    [1] 桑瑞婷. 面向高校迎新的机器人问答系统研究[硕士学位论文]. 重庆: 重庆理工大学, 2019.
    [2] 卢超. 基于深度学习的句子相似度计算方法研究[硕士学位论文]. 太原: 中北大学, 2019.
    [3] 徐雄. 基于深度学习的问答系统研究. 湖北师范大学学报(自然科学版), 2019, 39(1): 10–18
    [4] Kumar A, Irsoy O, Ondruska P, et al. Ask me anything: Dynamic memory networks for natural language processing. Proceedings of the 33rd International Conference on International Conference on Machine Learning. New York City, NY, USA. 2016. 1378–1387.
    [5] Wang Q, Xu CM, Zhou YM, et al. An attention-based Bi-GRU-CapsNet model for hypernymy detection between compound entities. Proceedings of 2018 IEEE International Conference on Bioinformatics and Biomedicine. Madrid, Spain. 2018. 1031–1035.
    [6] Dos Santos C, Tan M, Xiang B, et al. Attentive pooling networks. arXiv: 1602.03609, 2016.
    [7] Peters M E, Neumann M, Iyyer M, et al. Deep contextualized word representations. Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, LA, USA. 2018. 2227–2237.
    [8] Zhou XY, Dong DX, Wu H, et al. Multi-view response selection for human-computer conversation. Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing. Austin, TX, USA. 2016. 372–381.
    [9] Huang ZH, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. arXiv: 1508.01991, 2015.
    [10] 李琳, 李辉. 一种基于概念向量空间的文本相似度计算方法. 数据分析与知识发现, 2018, 2(5): 48–58
    [11] Wang BN, Liu K, Zhao J. Inner attention based recurrent neural networks for answer selection. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin, Germany. 2016. 1288–1297.
    [12] Graves A. Supervised Sequence Labelling. Berlin, Heidelberg: Springer. 2012. 5–13.
    [13] Adomavicius G, Zhang JJ. Classification, ranking, and top-k stability of recommendation algorithms. INFORMS Journal on Computing, 2016, 28(1): 129–147. [doi: 10.1287/ijoc.2015.0662
    [14] Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in vector space. arXiv: 1301.3781, 2013.
    [15] İrsoy O, Cardie C. Deep recursive neural networks for compositionality in language. Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, QC, Canada. 2014. 2096–2104.
    [16] 周艳平, 李金鹏, 蔡素. 基于同义词词林的句子语义相似度方法及其在问答系统中的应用. 计算机应用与软件, 2019, 36(8): 65–68
    [17] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 2014, 15(1): 1929–1958
    [18] He W, Liu K, Liu J, et al. Dureader: A chinese machine reading comprehension dataset from real-world applications. Proceedings of the Workshop on Machine Reading for Question Answering. Melbourne, Australia. 2017. 37–46.
    [19] Abadi M, Agarwal A, Barham P, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv: 1603.04467, 2016.
    [20] 赵明, 董翠翠, 董乔雪, 等. 基于BIGRU的番茄病虫害问答系统问句分类研究. 农业机械学报, 2018, 49(5): 271–276
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

周艳平,朱小虎.基于正负样本和Bi-LSTM的文本相似度匹配模型.计算机系统应用,2021,30(4):175-180

Copy
Share
Article Metrics
  • Abstract:1118
  • PDF: 2565
  • HTML: 2519
  • Cited by: 0
History
  • Received:July 30,2020
  • Revised:August 26,2020
  • Online: March 31,2021
Article QR Code
You are the first990398Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063