基于深度信念网络的文本分类算法
作者:

Text Categorization Based on Deep Belief Network
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [22]
  • |
  • 相似文献 [20]
  • |
  • 引证文献
  • | |
  • 文章评论
    摘要:

    随着网络的迅猛发展, 文本分类成为处理和组织大量文档数据的关键技术. 目前已经有许多不同类型的神经网络应用于文本分类, 并且取得良好的效果. 但是, 大部分模型仅采用文档的少量特征作为输入, 没有考虑到足够的信息量; 而当考虑到足够的特征时, 又会发生维数灾难, 导致模型难以训练或者训练时间大幅增加. 利用深度信念网络从文本中抽取特征, 并利用softmax回归分类器对抽取后的特征分类. 深度信念网络不仅具有强大的学习能力, 同时还能从高维的原始特征中抽取低维度高度可区分的低维特征, 因此利用深度信念网络来对文本分类, 不仅能够考虑到文档的足够的信息量, 而且能够快速的训练. 并且实验结果也表明利用深度信念网络实现文本分类的性能很好.

    Abstract:

    With the rapid development of the network, text categorization has become a key technology in processing and organizing large text. There are already many different types of neural networks applied to text categorization and achieve good results. However, most of the document models use only a small amount of features as input which don't take into account the sufficient amount of information. Considering enough characteristics, the dimensions of the disaster will occur, and resulting in a substantial increase training time, difficulty in training the model. This paper considers using deep belief networks to extract features from the text, then using softmax regression classifier for classification. The deep belief networks not only has a high learning ability, but also can extract highly distinguishable and low-dimensional features from the original high-dimensional features. So taking advantage of the deep belief networks to classify the text, can take into account enough information document amount. And the result shows that the use of deep belief networks achieve good performance for text categorization .

    参考文献
    1 McCallum A, Nigam K. A comparison of event models for naive bayes text classification. AAAI-98 Workshop on Learning for Text Categorization. Madison, Wisconsim(32).
    2 白莉媛,黄晖,刘素华,阎秋玲.基于自助平均的朴素贝叶斯文本分类器,2007,33(15):190-192.
    3 Joachims T. Text categorization with support vector machines: Learning with many relevant features. European Conference on Machine Learning (ECML). Chemnitz, Germany. 1998. 137-142.
    4 Ruiz ME, Srinivasan P. Hierarchical neural networks for text categorization. Proc. of SIGIR-99, 22nd ACM International Information Retrieval. 1999(32). 281-282.
    5 叶菲,罗景青,俞志富.一种改进的并行处理SVM学习算法.微电子学与计算机,2009,26(2):40-43.
    6 Guo GD, Wang H, Bell D, Bi YX, Greer KR. An kNN Model-based Approach and Its Application in Text Categorization. CICLing 2004, LNCS 2945, 2004. 559-570.
    7 吴春颖,王士同.一种改进的KNN Web文本分类方法.计算机应用研究,2008,25(11):3275-3277.
    8 曾砺锋.基于Rocchio方法和k均值聚类的支持向量机文本分类方法.软件导刊,2008,7(6):37-39.
    9 陈世立,高野军.基于神经网络与贝叶斯的混合文本分类研究.情报杂志,2007(5):34-36.
    10 Debole F, Scbastiani F. An analysis of the relative hardness of recuters-21578 subsets. Journal of the American Society for Information Science and Technology,2004,56(6): 584-596
    11 Bengio Y. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2009, 2(1): 1-127.
    12 孙志军,薛磊,许阳明,等.深度学习研究综述.计算机应用研究,2012,29(8):2806-2810.
    13 Bengio Y, Delalleau O. On the expressive power of deep architectures. Proc of the 14th International Conference on Discovery Science. Berlin: Springer-Verlag, 2011: 18-36.
    14 Bengio Y, Lecun Y. Scaling learning algorithms towards AI. In: Bottou L, Chapelle O, Decosted, et al. eds. Large-Scale Kernel Machines. Cambridge: MIT Press, 2007: 321-358.
    15 Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. Science, 2006, 313(5786): 504-507.
    16 Hinton GE, Osindero S, Teh Y. A fast learning algorithm for deep belief nets. Neural Computation, 2006, 18(7): 1527-1554.
    17 Lee H, Grosse R, Ranganath R, et al. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. Proc. of the 26th Annual International Conference on Machine Learning. ACM. 2009. 609-616.
    18 Dong Y, Li D. Deep learning and its applications to signal and information processing. IEEE Signal Process Mag, 2011, 28: 145-154.
    19 Fox C. Lexical analysis and stoplists. Information Retrieval: Data Structure and Algorithms, 1992, 7: 102-130.
    20 Frakes WB. Stemming algorithms. Information Retrieval: Data Structure and Algorithms, 1992, 3: 131-160.
    21 Roux NL, Bengio Y. Representational power of restricted Boltzmann machines and deep belief networks. Neural Computation, 2006, 20(6): 1631-1649.
    22 Cai D, He XF. Manifold adaptive experimental design for text categorization. IEEE Trans. on Knowledge and Data Engineering, 2011.
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

陈翠平.基于深度信念网络的文本分类算法.计算机系统应用,2015,24(2):121-126

复制
分享
文章指标
  • 点击次数:1925
  • 下载次数: 5528
  • HTML阅读次数: 0
  • 引用次数: 0
历史
  • 收稿日期:2014-06-17
  • 最后修改日期:2014-07-22
  • 在线发布日期: 2015-03-04
文章二维码
您是第11224692位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号