In recent years, the rapid development of deep learning has led more and more people to engage in related research work. However, many researchers construct deep neural network models based on standard algorithms or improved algorithms, but do not understand the algorithm itself and the factors that affect the performance of the model, resulting in more or less blind application in many applications. By studying the deep neural network, the activation function of the important influencing factors was studied. First, the activation function is analyzed to influence the depth neural network. Then, the development of activation function and the principle and performance of different activation functions are analyzed and summarized. Finally, based on the Caffe framework, the CNN is used to classify and identify MNIST data sets. Five kinds of commonly used activation functions are analyzed and compared comprehensively to provide a reference for the selection of activation function in the design of deep neural network model.