The clothing image classification model has many parameters, high time complexity, and low accuracy of clothing classification. In response, a network pruning method and network sparsity constraints were proposed to reduce the redundant parameters from the convolutional layer to the full connection layer in the Xception of the convolutional neural network (CNN), increase the sparsity and randomness of the network, reduce the over-fitting phenomenon, and lower the time complexity and computational complexity of the model as much as possible without affecting the accuracy. In addition, the attention mechanism SE-Net module is introduced into the convolution layer to improve the accuracy of clothing image classification. The experimental results on the DeepFashion dataset show that the network model reduced by the network pruning method has brought down both the spatial complexity and the time complexity. Compared with VGG-16, ResNet-50, and Xception models, the proposed model, with higher clothing image classification accuracy and operation efficiency, lowers the requirements on equipment and paves the way for the application of the deep CNN on mobile terminals and embedded devices. It has a high value in the application of e-commerce platforms in real-world clothing.