﻿ 基于模拟退火算法的改进极限学习机
 计算机系统应用  2020, Vol. 29 Issue (2): 163-168 PDF

Improved Extreme Learning Machine Based on Simulated Annealing Algorithm
WU Yu
School of Management, University of Science and Technology of China, Hefei 230026, China
Abstract: As a supervised learning model, traditional extreme learning machine assigns input weights and bias of nodes of hidden layer arbitrarily, and completes learning process by calculating output weights of nodes of hidden layer. Aiming at the problem that traditional extreme learning machine does not work well in prediction research, an improved extreme learning machine model based on simulated annealing algorithm was proposed. Firstly, traditional extreme learning machine method was used to learn the training set, and output weight of hidden layer is obtained. The evaluation standard of prediction result was selected to assess prediction result. Then, using the simulated annealing algorithm, input weights and bias of hidden layer of traditional extreme learning machine were regarded as the initial solution, and the evaluation standard was regarded as the objective function. The optimal solution was found in cooling process that was input weights and bias of hidden layer of extreme learning machine with the smallest prediction error. Iris classification data and Boston house price forecast data were used to conduct experiments. The experiment finds that compared with traditional extreme learning machine, extreme learning machine based on simulated annealing is better on classification and regression.
Key words: Extreme Learning Machine (ELM)     Simulated Annealing (SA)     classification prediction     regression prediction

1 基于SA改进的ELM预测模型 1.1 极限学习机

 $\begin{array}{l} H({W_1}, \cdots ,{W_L},{b_1}, \cdots ,{b_L},{x_1}, \cdots ,{x_N})\\ {\rm{ = }}{\left[ \begin{array}{ccc} g({W_1}*{x_1} + {b_1}) & \cdots & g({W_L}*{x_1} + {b_L})\\ \vdots & \ddots &\vdots\\ g({W_1}*{x_N} + {b_1}) & \cdots& g({W_L}*{x_N} + {b_L}) \end{array} \right]_{N \times L}} \end{array}$
 $\beta = {\left[ \begin{array}{l} \beta _1^{\rm{T}}\\ \; \vdots \\ \beta _L^{\rm{T}} \end{array} \right]_{L \times m}},Y = {\left[ \begin{array}{l} {{y}}_1^{\rm{T}}\\ \; \vdots \\ y_N^{\rm{T}} \end{array} \right]_{N \times m}}$

1.2 模拟退火算法

1)初始化, 取足够大的初始温度 $\scriptstyle {T_0}$ , 令 $\scriptstyle T = {T_0}$ , 任意选取初始解 $\scriptstyle{S_1}$ , 确定每个温度下的迭代次数L和结束温度 $\scriptstyle{{T}_{{\rm{end}}}}$ .

2)对当前温度T $\scriptstyle{k} = {{1,2,}} \cdots {{,L}}$ , 重复第3)步至第5)步.

3)随机产生一个新解 $\scriptstyle{S_2}$ .

4)计算 $\scriptstyle \Delta {f} = {f}({S_2}) - f({S_1})$ , 其中 $\scriptstyle{f}({S_1})$ , $\scriptstyle{f}(S_2)$ 分别是 $\scriptstyle{S_1}$ , $\scriptstyle{S_2}$ 的代价函数.

5)若 $\scriptstyle\Delta {{f < }}0$ , 则接受 $\scriptstyle{S_2}$ 作为新的当前解, 令 $\scriptstyle{S_1} = S_2$ ; 否则计算 $\scriptstyle{S_2}$ 的接受概率 $\scriptstyle{\exp}( - \Delta f/T)$ , 若产生(0,1)区间的随机数rand, $\scriptstyle{{\exp}}( - \Delta f/T) > {\rm{rand}}$ , 也接受 $\scriptstyle{S_2}$ 作为新的当前解, $\scriptstyle{S_1} = {S_2}$ ;否则当前解 $\scriptstyle{S_1}$ 不变.

6)降温速度为 $\scriptstyle{q}$ , $\scriptstyle{T} = {q*T}$ , 若T小于 $\scriptstyle{T_{\rm{end}}}$ , 则停止迭代, 输出当前解, 否则继续迭代.

1.3 改进的极限学习机

1)任意生成输入权值 $\scriptstyle W_{L \times N}^0$ 和偏置 $\scriptstyle{b}_L^0$ , 极限学习机学习训练集, 得到输出权值 $\scriptstyle{\hat \beta ^0}$ , 均方根误差为预测评价标准, 计算训练误差 $\scriptstyle{rms}e^0$ .

2)设置初始温度 $\scriptstyle{T} = T_0$ , 结束温度 $\scriptstyle{T_{{\rm{end}}}}$ , 每个温度下的迭代次数L, 降温速度 $\scriptstyle{q}$ , 令 $\scriptstyle{k} = 0$ .

3)任意生成输入权值 $\scriptstyle{W_{L \times N}}$ 和偏置 $\scriptstyle{b_L}$ , 并且 $\scriptstyle{W_{L \times N}} \ne W_{L \times N}^0$ , $\scriptstyle{b_L} \ne {{b}}_L^0$ . 极限学习机学习训练集, 得到输出权值 $\scriptstyle \;\hat \beta$ . 设置代价函数即均方根误差 $\scriptstyle rmse$ , $\scriptstyle k = k +1$ .

4)若 $\scriptstyle \Delta f = rmse - rms{e^0},\;\Delta f< 0$ 成立, 则 $\scriptstyle W_{L \times N}^0 = {W_{L \times N}}$ , $\scriptstyle{b}_L^0 = {b_L}$ ; 若不成立, 则计算输入权值 $\scriptstyle{W_{L \times N}}$ 和偏置 $\scriptstyle{b_L}$ 的接受概率 $\scriptstyle{exp}( - \Delta {{f}}/T)$ , 并且生成一个(0, 1)区间的随机数rand, 若 $\scriptstyle{exp}( - \Delta {{f}}/T) > {\rm{rand}}$ 成立, 则 $\scriptstyle W_{L \times N}^0 = {W_{L \times N}}$ , $\scriptstyle{b}_L^0 = {b_L}$ ; 否则返回第3)步.

5)若 $\scriptstyle k > L$ 成立, 则 $\scriptstyle T = T*q$ , $\scriptstyle k = 0$ , 否则返回第3)步.

6)若 $\scriptstyle T = {T_{end}}$ 不成立, 则返回第3)步, 若成立, 则算法结束, 得到最优的输入权值 $\scriptstyle W_{L \times N}^0$ 和偏置 $\scriptstyle{b}_L^0$ .

2 实验分析

2.1 鸢尾花分类

 图 1 改进的极限学习机分类的优化过程

2.2 波士顿房价预测

3 结论与展望

 图 2 极限学习机的波士顿房价预测结果

 图 3 改进的极限学习机的波士顿房价预测结果

 图 4 改进的极限学习机回归的优化过程

 [1] Tang JX, Deng CW, Huang GB. Extreme learning machine for multilayer perceptron. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(4): 809-821. DOI:10.1109/TNNLS.2015.2424995 [2] 常玉清, 李玉朝, 王福利, 等. 基于极限学习机的生化过程软测量建模. 系统仿真学报, 2007, 19(23): 5587-5590. DOI:10.3969/j.issn.1004-731X.2007.23.057 [3] 王杰, 毕浩洋. 一种基于粒子群优化的极限学习机. 郑州大学学报(理学版), 2013, 45(1): 100-104. [4] 王莉, 陈红梅, 王生武. 新的基于代价敏感集成学习的非平衡数据集分类方法NIBoost. 计算机应用, 2019, 39(3): 629-633. DOI:10.11772/j.issn.1001-9081.2018071598 [5] 郑仙花, 骆炎民. 基于多类数据分类的改进克隆选择算法. 计算机应用, 2012, 32(11): 3201-3205. [6] 沈宋衍, 陈莹. 基于在线回归学习的轮廓跟踪算法. 计算机工程, 2016, 42(5): 230-234. DOI:10.3969/j.issn.1000-3428.2016.05.039 [7] 王英博, 聂娜娜, 王铭泽, 等. 修正型果蝇算法优化GRNN网络的尾矿库安全预测. 计算机工程, 2015, 41(4): 267-272. DOI:10.3969/j.issn.1000-3428.2015.04.051 [8] 蒙凯, 唐秋华, 张子凯, 等. 基于改进多目标灰狼算法的装配线平衡与预防维护集成优化. 计算机集成制造系统. https://www.cnki.net/KCMS/detail/11.5946.tp.20190812.1025.016.html, [2019-08-18]. [9] 赵燕伟, 任设东, 陈尉刚, 等. 基于改进BP神经网络的可拓分类器构建. 计算机集成制造系统, 2015, 21(10): 2807-2815. [10] 陈于思, 孙林夫. 基于差分进化和极限学习机的并发查询性能预测. 计算机集成制造系统, 2019, 25(9): 2291-2304. [11] 周华平, 袁月. 改进鱼群算法优化的ELM在乳腺肿瘤辅助诊断中的应用研究. 计算机工程与科学, 2017, 39(11): 2145-2152. DOI:10.3969/j.issn.1007-130X.2017.11.025 [12] 赵建堂. 大数据分割式极限学习机算法. 计算机工程与应用, 2019, 55(10): 73-76. DOI:10.3778/j.issn.1002-8331.1801-0217 [13] 郁磊, 史峰, 王辉, 等. MATLAB智能算法30个案例分析. 2版. 北京: 北京航空航天大学出版社, 2015. 178–187. [14] Mafarja MM, Mirjalili S. Hybrid Whale Optimization Algorithm with simulated annealing for feature selection. Neurocomputing, 2017, 260: 302-312. DOI:10.1016/j.neucom.2017.04.053 [15] 高鹰, 谢胜利. 基于模拟退火的粒子群优化算法. 计算机工程与应用, 2004, 40(1): 47-50. DOI:10.3321/j.issn:1002-8331.2004.01.014 [16] 杨若黎, 顾基发. 一种高效的模拟退火全局优化算法. 系统工程理论与实践, 1997, 17(5): 30-36. [17] 张世睿, 李心科. 基于模拟退火的BP网络隐藏层节点估算算法. 合肥工业大学学报(自然科学版), 2017, 40(11): 1489-1491, 1506. [18] 凌静, 江凌云, 赵迎. 结合模拟退火算法的遗传K-Means聚类方法. 计算机技术与发展, 2019, 29(9): 61-65. [19] 黄联标, 敖银辉. 基于模拟退火算法的多工段系统预防维护策略. 计算机集成制造系统. https://www.cnki.net/KCMS/detail/11.5946.tp.20190426.1318.012.html. [2019-04-29]. [20] 罗庚合. 基于可拓聚类的极限学习机神经网络. 计算机应用, 2013, 33(7): 1942-1945. DOI:10.11772/j.issn.1001-9081.2013.07.1942 [21] 白丹宇. 流水车间与开放车间调度算法渐近分析. 北京: 清华大学出版社, 2015. 6–7.