Plastic mobile phone shell factory quality inspection. It’s time-consuming and uses traditional manual methods to identify appearance defects. The classifier uses the convolution neural network model of deep learning to train. This classifier can automatically detect the scratch defects on the appearance of mobile phone shell, which can greatly improve the work efficiency. In the experiment, a basic convolution neural network model is established firstly, the recognition baseline is obtained by training model, and then the detection accuracy is gradually improved by design modification. In order to solve the model over fitting and improve the detection accuracy in small data set training, the dropout, data augmentation method and batch normalization are used to reduce the amount of parameters, and the transfer learning method is applied. Experimental results show that the classifier model can effectively improve the accuracy and achieve a very good scratch defect recognition effect on small data sets.