﻿ 基于自编码算法的深度学习综述
 计算机系统应用  2018, Vol. 27 Issue (9): 47-51 PDF

Overview on Deep Learning Based on Automatic Encoder Algorithms
CUI Guang-Xin, LI Dian-Kui
College of Information Science and Electronic Technology, Jiamusi University, Jiamusi 154007, China
Foundation item: Specific Foundation for Education and Scientific Research of Heilongjiang Province in 2017 (2017-0001)
Abstract: Deep learning is a branch of machine learning, creating a new era in the development of neural networks. As an important part of deep learning structure, self-coding algorithm plays a crucial role in unsupervised learning and nonlinear feature extraction. Firstly, the basic concepts and principles of self-encoding algorithm are introduced. Then, the improved algorithm based on self-encoding algorithm is presented. Finally, the well-known cases and development trends of self-encoding algorithm applied in several fields are elaborated.
Key words: machine learning     deep learning     automatic encoder     unsupervised learning     neural networks

1 自编码算法

 $h = f(x) = {s_f}(wx + p)$ (1)

 $y = g(h) = {s_g}(\tilde w h + q)$ (2)

${s_g}$ 为恒等函数时:

 $L(x,y) = ||x - y|{|^2}$ (3)

${s_g}$ 为Sigmoid函数时:

 $L(x,y) = - \sum\limits_{i = 1}^n {[{x_i}\log {y_i} + (1 - {x_i})\log (1 - {y_i})]}$ (4)

 ${J_{AE}}(\theta ) = \sum\limits_{x \in S} {L(x,g(f(x)))}$ (5)

2 基于自编码算法的改进算法 2.1 稀疏自编码算法

2.2 降噪自编码算法

2.3 收缩自编码算法

2.4 栈式自编码算法

2.5 边缘降噪自编码算法

2.6 稀疏降噪自编码算法

2.7 稀疏边缘降噪自编码算法

2.8 卷积自编码算法

2.9 卷积稀疏自编码算法

3 自编码算法的应用案例 3.1 图像识别领域

3.2 语音识别领域

3.3 故障诊断领域

3.4 其它领域

4 结论与展望

 [1] Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature, 1986, 323(9): 533-536. [2] Baldi P, Guyon G, Dror V, et al. Autoencoders, unsupervised learning, and deep architectures. Journal of Machine Learning Research, 2012, 27: 37-50. [3] Luo Y, Wan Y. A novel efficient method for training sparse auto-encoders. 2013 6th International Congress on Image and Signal Processing (CISP 2013). Hangzhou, China. 2013. 1019–1023. [doi: 10.1109/CISP.2013.6745205] [4] 史彩娟, 阮秋琦. 基于增强稀疏性特征选择的网络图像标注. 软件学报, 2015, 26(7): 1800-1811. DOI:10.13328/j.cnki.jos.004687 [5] Vincent P, Larochelle H, Bengio Y, et al. Extracting and composing robust features with denoising autoencoders. Proceedings of the 25th International Conference on Machine Learning. ACM. New York, NY, USA. 2008. 1096–1103. [doi: 10.1145/1390156.1390294] [6] Rifai S, Vincent P, Muller X, et al. Contractive auto-encoders: Explicit invariance during feature extraction. ProceedingICML’11 Proceedings of the 28th International Conference on International Conference on Machine Learning. Bellevue, WA, USA. 2011. 833-840. [7] Chen M, Weinberger K, Sha F, et al. Marginalized denoising auto-encoders for nonlinear representations. Proceedings of the 31st International Conference on Machine Learning. Beijing, China. 2014. 1476–1484. [8] 徐文瀚, 程石磊. 基于深度栈式自编码网络的面部年龄识别. 信息通信, 2017(11): 45-47. DOI:10.3969/j.issn.1673-1131.2017.11.020 [9] 李江, 冉君军, 张克非. 一种基于降噪自编码器的人脸表情识别方法. 计算机应用研究, 2016, 33(12): 3843-3846. DOI:10.3969/j.issn.1001-3695.2016.12.072 [10] 余涛. 基于稀疏自编码器的手写体数字识别. 数字技术与应用, 2017(1): 66-68. [11] 王飞飞. 基于稀疏自动编码器的近重复视频检索. 电子技术与软件工程, 2017(3): 194-196. [12] 王勇, 赵俭辉, 章登义, 等. 基于稀疏自编码深度神经网络的林火图像分类. 计算机工程与应用, 2014, 50(24): 173-177. DOI:10.3778/j.issn.1002-8331.1401-0392 [13] 马勇, 鲍长春. 基于稀疏神经网络的说话人分割. 北京工业大学学报, 2015, 41(5): 662-667. [14] 朱芳枚, 赵力, 梁瑞宇, 等. 面向中文语音情感识别的改进栈式自编码结构. 东南大学学报, 2017, 47(4): 631-636. [15] 黄丽霞, 王亚楠, 张雪英, 等. 基于深度自编码网络语音识别噪声鲁棒性研究. 计算机工程与应用, 2017, 53(13): 49-54. DOI:10.3778/j.issn.1002-8331.1611-0217 [16] 王辉, 赵悦, 刘晓凤, 等. 基于深度特征学习的藏语语音识别. 东北师大学报, 2015, 47(4): 69-73. [17] 王山海, 景新幸, 杨海燕. 基于深度学习神经网络的孤立词语音识别的研究. 计算机应用研究, 2015, 32(8): 2289-2291, 2298. DOI:10.3969/j.issn.1001-3695.2015.08.011 [18] 陈仁祥, 黄鑫, 杨黎霞, 等. 加噪样本扩展深度稀疏自编码神经网络的滚动轴承寿命阶段识别. 振动工程学报, 2017, 30(5): 874-882. [19] 温博文, 董文瀚, 解武杰, 等. 深度自编码观测器飞机操纵面快速故障诊断. 飞行力学, 2016, 34(6): 34-38, 44. [20] 刘辉海, 赵星宇, 赵洪山, 等. 基于深度自编码网络模型的风电机组齿轮箱故障检测. 电工技术学报, 2017, 32(17): 156-163. [21] 钱彬, 唐振民, 徐威. 基于稀疏自编码的路面裂缝检测. 北京理工大学学报, 2015, 35(8): 800-804, 809. [22] 朱煜奇, 黄双喜, 杨天祺, 等. 基于栈式降噪自编码的故障诊断. 制造业自动化, 2017, 39(3): 152-156. [23] 袁静, 章毓晋. 融合梯度差信息的稀疏去噪自编码网络在异常行为检测中的应用. 自动化学报, 2017, 43(4): 604-610. [24] 高妮, 高岭, 贺毅岳, 等. 基于自编码网络特征降维的轻量级入侵检测模型. 电子学报, 2017, 45(3): 730-739. DOI:10.3969/j.issn.0372-2112.2017.03.033 [25] 杨辉华, 雒志超, 蒋淑洁, 等. 稀疏降噪自编码算法用于近红外光谱鉴别真假药的研究. 光谱学与光谱分析, 2016, 36(9): 2774-2779. [26] 魏扬威, 黄萱菁. 结合语言学特征和自编码器的英语作文自动评分. 计算机系统应用, 2017, 26(1): 1-8. DOI:10.15888/j.cnki.csa.005535