﻿ LSTM网络模型在Web服务器资源消耗预测中的应用研究
 计算机系统应用  2019, Vol. 28 Issue (7): 214-220 PDF
LSTM网络模型在Web服务器资源消耗预测中的应用研究

Application of LSTM Network Model in Web Server Resource Consumption Prediction
TAN Yu-Ning, DANG Wei-Chao, BAI Shang-Wang, PAN Li-Hu
School of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan 030024, China
Foundation item: Science and Technology Collaborative Program between Shanxi Province and Chinese Academy of Sciences (20141101001); Key Research and Development Program for General Industry of Shanxi Province (201703D121042-1); Science and Technology Program for Social Development of Shanxi Province (20140313020-1)
Abstract: How to predict the aging trend of the software accurately, and take the corresponding recovery strategy is a key problem of preventing software aging. To solve the problem, this study designs a resource prediction method based on Recurrent Neural Network (RNN) and its variant–Long Short-Term Memory (LSTM), and builds an accelerated aging test platform to model and forecast the aging phenomenon of the Web server due to memory leak. The experiments show that LSTM network prediction model proves to be superior to the other traditional models in dealing with the time sequence modeling of aging parameters, with the predicted results closer to the actual values and the higher prediction accuracy, which can effectively improve the reliability and availability of the software system.
Key words: software aging     LSTM     resource consumption     Recurrent Neural Network (RNN)     software reliability

1 模型原理 1.1 循环神经网络

RNN是一类由各神经元相互连接形成的有向循环人工神经网络, 其基于时序展开后的结构如下图1所示. 与传统的前馈神经网络(Feedforward Neural Network, FNN)不同, RNN不仅通过层与层间的连接进行信息的传递, 而且通过在网络中引入环状结构, 建立了神经元到自身的连接. 每一步的输出不仅包括当前所见的输入样例, 还包括网络在上一个时刻所感知到的信息即当前时刻的ht不仅仅取决于当前时刻的输入xt, 而且与上一时刻的ht–1也相关.

 图 1 按照时序展开的RNN结构图

 ${{{h}}_t} = f\left( {U \cdot {x_t} + W \cdot {h_{t - 1}} + {b_h}} \right)$ (1)
 ${{\rm{o}}_t} = g\left( {V \cdot {h_t} + {b_o}} \right)$ (2)

 $\begin{split} o_{t}=&V \cdot g(U \cdot x_{t}+W \cdot g(U \cdot x_{t-1}+W \cdot g(U \cdot x_{t-2}+\\ &W \cdot g(U \cdot x_{t-3}+\cdots)))) \end{split}$ (3)

1.2 长短时记忆单元

 图 2 LSTM单元结构图

 $\left\{ {\begin{array}{*{20}{l}} {{{{f}}_{{t}}} = \sigma \left( {{W_f} \cdot {h_{t - 1}} + {W_f} \cdot {x_t} + {b_f}} \right)}\\ {{{{i}}_{{t}}} = \sigma \left( {{W_i} \cdot {h_{t - 1}} + {W_i} \cdot {x_t} + {b_i}} \right)}\\ {{{{o}}_t} = \sigma \left( {{W_o} \cdot {h_{t - 1}} + {W_o} \cdot {x_t} + {b_o}} \right)} \end{array}} \right.$ (4)

 $\left\{ {\begin{array}{*{20}{l}} {{{{h}}_t} = {o_t} \odot {\rm{tanh}}\left( {{c_t}} \right)}\\ {{{\tilde {{c}}}_t} = {\rm{tanh}}\left( {{W_c} \cdot {h_{i - 1}} + {W_c} \cdot {x_t} + {b_c}} \right)}\\ {{{{c}}_t} = {f_t} \odot {c_{t - 1}} + {i_t} \odot {{\tilde c}_t}} \end{array}} \right.$ (5)

1.3 LSTM预测模型具体构建过程

Web服务器资源消耗预测就是根据前t时刻老化指标的资源使用特征来预测t+1或者t+x时间内的资源损耗, 以此判断Web服务器的老化状况. 因此通过使用老化数据对LSTM神经网络进行训练, 构建基于LSTM网络的软件老化资源预测模型, 其具体构建步骤如下所示:

(1)首先将原始老化数据清洗后进行特征表示和特征提取: 定义老化资源损耗时间序列F={f1, f2, …, f}, 将其划分为测试集Ftrain={f1, f2, …, fm}和训练集Ftest={fm+1, fm+2, …, fn}, 其中m<nm, nN, 对Ftrain集合中的元素max-min标准化, 处理后的训练集表示为式(6).

 $\begin{array}{*{20}{l}} {F_{tr}^{'} = \left\{ {f_1^{'} ,f_2^{'} , \cdots ,f_m^{'} } \right\}}\\ {f_t^{'} = \left( {{f_t} - \mathop {\min \left\{ {{f_i}} \right\}}\limits_{1 \le j \le m} } \right)/\left( {\mathop {\max \left\{ {{f_j}} \right\}}\limits_{1 \le j \le m} - \mathop {\min \left\{ {{f_j}} \right\}}\limits_{1 \le j \le m} } \right)} \end{array}$ (6)

(2)构建训练输入以及对应的训练输出: 对数据集进行时间融合, 按照滑动窗口的大小s进行分割, 则模型输入、输出分别变为式(7)、式(8). 通过设置s的值, 旨在训练LSTM网络学习样本数据前后的关联及规律.

 $\left\{ \begin{array}{*{20}{l}} {X = \left\{ {{X_1},{X_2}, \cdots ,{X_S}} \right\}}\\ {{X_{{p}}} = \left\{ {f_p^{'} ,f_{p + 1}^{'} , \cdots ,{f_{m - {\rm{s}} + p - 1}^{'}}} \right\}} \end{array}\right.$ (7)
 $\left\{ \begin{array}{l} Y = \left\{ {{Y_1},{Y_2}, \cdots ,{Y_S}} \right\}\\ {Y_{\rm{p}}} = \left\{ {f_{p + 1}^{'} ,f_{p + 2}^{'} , \cdots ,f_{m - t + p}^{'} } \right\} \end{array}\right.$ (8)

(3)确定网络结构并初始化LSTM网络: 确定每层激活函数的选择、每层网络节点的舍弃率以及误差计算方式、权重参数迭代的更新方式. 给定初始权值矩阵, 设置最大迭代训练次数和最小误差值, 通过改变网络的各项参数来训练网络.

(4)前向计算: 将X输入网络, 根据前向计算公式(4)–(5)计算遗忘门、输入门以及输出门的值, 经过隐藏层后的输出结果可表示为: 其中CP–1HP–1分别表示上一个LSTM细胞的状态以及隐含层的输出.

 $\left\{ {\begin{array}{*{20}{l}} {P = \left\{ {{P_1},{P_2}, \cdots ,{P_L}} \right\}}\\ {{P_p} = LST{M_f}\left( {{X_p},{C_{p - 1}},{H_{p - 1}}} \right)} \end{array}} \right.$ (9)

(5)误差反向传播: 采用批量梯度下降算法对训练数据进行批次(batch)划分, 通过对当前批次的损失函数进行优化, 实时调整LSTM网络的权值和偏置, 使网络误差不断减小, 既保证了参数的更新又减少了模型收敛所需要的迭代次数.

 $loss = \sum\limits_{i = 1}^{L(m - L)} {{{\left( {{p_i} - {y_i}} \right)}^2}} /(L(m - L))$ (10)

(6)将训练好的模型用于预测: 当迭代次数和最小误差值满足要求时停止训练模型, 并将未知的样本数据通过迭代输入模型得到预测序列Pte={pm+1, pm+2, …, pn}, 并对其进行反标准化处理得到最终预测序如式11.

 $\left\{ \begin{array}{l} P_{te}^{'} = Anti\left( {{P_{te}}} \right) = \left\{ {p_{m + 1}^{'} ,p_{m + 2}^{'} , \cdots ,p_n^{'} } \right\}\\ p_k^{'} = {p_k}\left( {\mathop {\max \left\{ {{f_j}} \right\}}\limits_{1 \le j \le m} - \mathop {\min \left\{ {{f_j}} \right\}}\limits_{1 \le j \le m} } \right) + \mathop {\min \left\{ {{f_j}} \right\}}\limits_{1 \le j \le m} \end{array}\right.$ (11)

 $MAE = \frac{1}{n}\sum\limits_{i = 1}^n {\left| {{p_t} - {y_t}} \right|}$ (12)
 $RMSE = \sqrt {\frac{1}{n}\sum\limits_{i = 1}^n {{{\left( {{p_t} - {y_t}} \right)}^2}} }$ (13)

2 实验与结果分析

2.1 Web软件老化加速寿命实验

2.2 LSTM资源消耗预测建模

2.3 实验分析

 图 3 实验数据

 图 4 实验预测结果图

 图 5 实验损失函数图

 图 6 实验对比结果预测图

3 结束语

 [1] Yan YQ. Variance analysis of software ageing problems. IET Software, 2018, 12(1): 41-48. DOI:10.1049/iet-sen.2016.0290 [2] Kolettis N, Fulton N D. Software rejuvenation: Analysis, module and applications. Twenty-Fifth International Symposium on Fault-Tolerant Computing. Washington, DC, USA, 1995. 381–390. [3] Torquato M, Araujo J, Umesh I M, et al. SWARE: A methodology for software aging and rejuvenation experiments. Journal of Information Systems Engineering & Management, 2018, 3(2): 15. [4] 梁佩. 基于WEB技术的软件老化趋势预测研究[硕士学位论文]. 哈尔滨: 哈尔滨工程大学, 2013. [5] 苏莉, 齐勇, 金玲玲, 等. 基于非线性多参数模型的软件老化检测. 计算机科学, 2013, 40(1): 161-165, 170. DOI:10.3969/j.issn.1002-137X.2013.01.037 [6] Jia SQ, Hou CY, Wang JS. Software aging analysis and prediction in a web server based on multiple linear regression algorithm. 2017 IEEE 9th International Conference on Communication Software and Networks. Guangzhou, China. 2017. 1452–1456. [7] 渊岚. 基于人工神经网络的软件系统老化预测研究[硕士学位论文]. 西安: 西安建筑科技大学, 2016. [8] 闫永权, 郭平. 使用混合模型预测Web服务器中的资源消耗. 计算机科学, 2016, 43(10): 47-52. DOI:10.11896/j.issn.1002-137X.2016.10.008 [9] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553): 436-444. DOI:10.1038/nature14539 [10] 梁天新, 杨小平, 王良, 等. 记忆神经网络的研究与发展. 软件学报, 2017, 28(11): 2905-2924. DOI:10.13328/j.cnki.jos.005334 [11] Sak H, Senior A, Beaufays F. Long short-term memory recurrent neural network architectures for large scale acoustic modeling. Fifteenth Annual Conference of the International Speech Communication Association. Singapore. 2014. 338–342. [12] Cortez B, Carrera B, Kim Y J, et al. An architecture for emergency event prediction using LSTM recurrent neural networks. Expert Systems with Applications, 2018, 97: 315-324. DOI:10.1016/j.eswa.2017.12.037 [13] Zhao J, Jin YL, Trivedi KS, et al. Software rejuvenation scheduling using accelerated life testing. ACM Journal on Emerging Technologies in Computing Systems, 2014, 10(1): 1-23. [14] Yin YC, Coolen FPA, Coolen-Maturi T. An imprecise statistical method for accelerated life testing using the power-Weibull model. Reliability Engineering & System Safety, 2017, 167: 158-167. [15] Bezenek T, Cain T, Dickson R, et al. TPC-WBenchmark. http://pharm.ece.wisc.edu/tpcw.shtml. [16] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 2014, 15(1): 1929-1958.