In order to solve the problem that the existing facial retouching detection algorithm has complex feature extraction and low recognition rate, we proposed a facial retouching detection algorithm based on Multi-Scale-Convolutional Neural Network (MS-CNN). Different from the traditional CNN, MS-CNN adds image preprocessing, which uses Histograms of Oriented Gradient (HOG) feature-based facial extraction algorithm to extract the facial part from the original image. It connects the Local Response Normalization (LRN) layer after the first pooling layer to accelerate the convergence of the model. A multi-scale convolution layer is proposed, which cascades convolution kernel of 1×1, 3×3, and 5×5 to improve the classification accuracy. The experimental results show that the detection accuracy of the proposed algorithm is 99.5% in LFW data set and 92.9% in ND-ⅢTD data set, respectively. Compared with the mainstream network structure and existing facial retouching detection algorithms, the detection accuracy of the proposed algorithm is significantly improved.