本文已被:浏览 1858次 下载 5164次
Received:March 21, 2017 Revised:April 13, 2017
Received:March 21, 2017 Revised:April 13, 2017
中文摘要: 文本相似度主要应用于学术论文查重检测、搜索引擎去重等领域,而传统的文本相似度计算方法中的特征项提取与分词环节过于冗杂,而且元素的随机挑选也会产生权重的不确定性. 为了解决传统方法的不足,提出一种基于改进的Jaccard系数确定文档相似度的方法,该算法综合考虑了各元素、样本在文档中的权重及其对多个文档相似度的贡献程度. 实验结果表明,基于改进的Jaccard系数的文档相似度算法具有实效性并且能够得到较高的准确率,适用于各种长度的中英文文档,有效地解决现有技术中存在的文档间相似度计算不精的问题.
Abstract:Text similarity check is mainly used in Re-check detection of Papers, the deduplication of search engines and other fields. However, it's extremely fussy to extract feature items with the traditional methods for computing the text similarity. In addition, it will bring uncertainty to select elements randomly. To solve these problems, a text similarity method based on improved Jaccard coefficient is proposed. This method takes into account the weights of elements and samples in the document, even the contribution degree to multiple text similarity. The results suggest that the text similarity method based on the improved Jaccard coefficient has been proved to be effective with a satisfactory accuracy, which can be applicable to various lengths of Chinese, English documents. It effectively solves the problem of inexact computing with existing technologies.
文章编号: 中图分类号: 文献标志码:
基金项目:国家自然科学基金(61472082);福建省自然科学基金(2014J01220)
引用文本:
俞婷婷,徐彭娜,江育娥,林劼.基于改进的Jaccard系数文档相似度计算方法.计算机系统应用,2017,26(12):137-142
YU Ting-Ting,XU Peng-Na,JIANG Yu-E,LIN Jie.Text Similarity Method Based on the Improved Jaccard Coefficient.COMPUTER SYSTEMS APPLICATIONS,2017,26(12):137-142
俞婷婷,徐彭娜,江育娥,林劼.基于改进的Jaccard系数文档相似度计算方法.计算机系统应用,2017,26(12):137-142
YU Ting-Ting,XU Peng-Na,JIANG Yu-E,LIN Jie.Text Similarity Method Based on the Improved Jaccard Coefficient.COMPUTER SYSTEMS APPLICATIONS,2017,26(12):137-142