Considering the growing development of e-commerce platforms, the use of artificial classification of clothing classification cannot meet the current needs. In this study, based on the actual application scenarios, the design is improved in three aspects:the interference of background factors, the key position information of the garment image, and the hardware requirements of the algorithm model operation when classifying the garment image. Accordingly, it is proposed that to remove background interference, to use of local information of images, and to lightweight processing of the model. Finally, on the premise of satisfying the accuracy, the algorithm model that can be operated in the ordinary low-configuration PC terminal is obtained, which improves the work efficiency and saves the cost.