﻿ 基于自动驾驶系统的轻量型卷积神经网络优化
 计算机系统应用  2020, Vol. 29 Issue (3): 93-99 PDF

Optimization of Lightweight Convolution Neural Network Based on Automatic Driving System
GAO Xiu-Long, GE Dong-Yuan
School of Mechanical and Transportation Engineering, Guangxi University of Science and Technology, Liuzhou 545000 china
Foundation item: National Natural Science Foundation of China (51765007); Natural Science Foundation of Guangxi (2016GXNSFAA380111)
Abstract: Computer vision technology is widely used in autopilot system, which mainly solves the problem of object recognition and object classification. In this study, a lightweight neural network structure is proposed according to the task. In order to solve the problem of insufficient training data, an improved data enhancement algorithm is adopted to double the training data. At the same time, in order to solve the problem of using data generator as verification set and unable to use tensorboard, a solution is proposed. The principle of neural network processing image information is studied in detail by convolution network visualization method, and the optimization method is put forward. The accuracy of the trained model is 97.5% on the verification set, which meets the accuracy needs of autopilot system for classification task.
Key words: computer vision     classification network     data enhancement     convolution network visualization

1 轻量级网络架构

1.1 卷积层设计

1.2 全连接层设计

2 神经网络优化 2.1 激活函数的选择

Tanh激活函数[6]: 当训练数据分布较大或较小时, 函数梯度接近于0, 易造成梯度消失, 故应用Tanh函数时, 要先将训练数据进行归一化. Tanh函数如:

 $f(x) = \frac{{1 - \mathop e\nolimits^{ - 2x} }}{{1 + \mathop e\nolimits^{2x} }}$ (1)

Relu激活函数: 当训练数据分布大于0时, 其梯度不会消失且易于计算, 当训练数据分布小于0时, 神经元不被激活, 这种特性有助于网络稀疏性的建立, Relu函数如:

 $f(x) = \max (x,0)$ (2)

 图 1 激活函数对比

2.2 梯度下降算法的选择与优化

SGD[7,8]算法运算速度快并且占用硬件资源少, 是当前深度学习领域主要的优化算法, 公式为:

 $g(\varphi ) = \displaystyle\sum\limits_{j = 0}^n {\mathop \varphi \nolimits_j } \mathop x\nolimits_j$ (3)
 $h(\varphi ) = \frac{1}{{2m}}\sum\limits_{i = 1}^m {(\mathop y\nolimits^i - \mathop g\nolimits_\varphi (\mathop x\nolimits^i )\mathop )\nolimits^2 }$ (4)
 $\varphi = \varphi - \eta \mathop \nabla \nolimits_\varphi h(\varphi )$ (5)

 图 2 学习率测试

2.3 过拟合问题处理

Dropout正则化: 本文在全连接层中使用Dropout[11,12]算法, 该算法能够使全连接层中的一部分神经元以0.5的概率失活, 失活的神经元激活函数输出为0. 此时神经元在网络中没有任何贡献. 在测试阶段在乘以一定的系数并重新激活, 这种正则化方法由于其随机性, 可以为网络引入一些随机噪声, 这样可以使模型的泛化能力变强, 因为它不会太依赖某些局部的特征, 从而减少模型的过拟合, 如图3(a)是没有引入正则化与Dropout处理方法而进行的分类网络实验, 可以看到模型过度拟合了数据, 包括噪音, 而图3(b)是引入权值正则化和Dropout的网络, 可看到网络自动将两个红色的噪音屏蔽掉, 从而使训练出的模型分类准确率得到提高.

3 实验结果与分析 3.1 实验平台的搭建

 图 3 随机失活测试

3.2 数据收集与数据增强

 图 4 数据增强

3.3 实验验证

if python_version_tuple() [0] == '3':

xrange = range

imap = map

else:

x, y = izip (*(validation_generator[i] for i in xrange

(len(validation_generator))))

x_val, y_val = np.vstack(x), np.vstack(y)

 图 5 准确率与损失值变化曲线

 图 6 卷积层权值与偏置值分布变化

 图 7 全连接层权值与偏置值分布变化

4 卷积神经网络的可视化 4.1 可视化卷积神经网络的特征输出

 图 8 卷积层可视化测试

4.2 类激活热力图

 图 9 热力激活测试

5 结束语

 [1] 林景栋, 吴欣怡, 柴毅, 等. 卷积神经网络结构优化综述. 自动化学报, 2019, 1-14. [2] 高磊, 范冰冰, 黄穗. 基于残差的改进卷积神经网络图像分类算法. 计算机系统应用, 2019, 28(7): 139-144. DOI:10.15888/j.cnki.csa.006997 [3] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, NV, USA. 2012. 1097–1105. [4] Wang ZQ, Liu J. A review of object detection based on convolutional neural network. Proceedings of the 36th Chinese Control Conference. Dalian, China. 2017. 11104–111092. [5] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553): 436-444. DOI:10.1038/nature14539 [6] 田娟, 李英祥, 李彤岩. 激活函数在卷积神经网络中的对比研究. 计算机系统应用, 2018, 27(7): 43-49. DOI:10.15888/j.cnki.csa.006463 [7] 王功鹏, 段萌, 牛常勇. 基于卷积神经网络的随机梯度下降算法. 计算机工程与设计, 2018, 39(2): 441-445, 462. [8] 史加荣, 王丹, 尚凡华, 等. 随机梯度下降算法研究进展. 自动化学报, 2019. DOI:10.16383/j.aas.c190260 [9] 蒋文斌, 彭晶, 叶阁焰. 深度学习自适应学习率算法研究. 华中科技大学学报(自然科学版), 2019, 47(5): 79-83. [10] Wan L, Zeiler M, Zhang SX, et al. Regularization of neural networks using dropconnect. Proceedings of the 30th International Conference on Machine Learning. Atlanta, GA, USA. 2013. 1058–1066. [11] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 2014(15): 1929-1958. [12] Wager S, Wang SD, Liang P. Dropout training as adaptive regularization. Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe, NV, USA. 2013. 351–359. [13] 吴睿曦, 肖秦琨. 基于深度网络和数据增强的多物体图像识别. 国外电子测量技术, 2019, 38(5): 86-90. [14] 俞海宝, 沈琦, 冯国灿. 在反卷积网络中引入数值解可视化卷积神经网络. 计算机科学, 2017, 44(S1): 146-150. [15] Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland. 2014. 818–833. [16] Yosinski J, Clune J, Nguyen A, et al. Understanding neural networks through deep visualization. arXiv: 1506.06579, 2015. [17] Erhan D, Bengio Y, Courville A, et al. Visualizing higher-layer features of a deep network. Technical Report 1341. Montreal: University of Montreal, 2009. [18] Selvaraju RR, Cogswell M, Das A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy. 2017. 618–626.