Economic globalization has given logo a huge commercial value, and the development of the computer vision provides a broader application field for logo classification and recognition. This study considers that the overall features of logo images are not significant, and the number of images is large, and then it proposes progressive multi-granularity training of jigsaw patches (PMG-Net), a method of fine-grained image classification, to classify the logo image dataset, so as to improve the ability of the model to classify logo images. The input images containing different granularity information are generated by the jigsaw patch generator, and then the progressive multi-granularity training module is introduced to fuse the features of different granularities. The fused features pay more attention to the subtle differences between images so that the effect of logo image classification is significantly improved. The leaky rectified linear unit (LeakyReLU) activation function is used to retain the negative feature information in the image when the input image features are extracted, and the channel attention mechanism is introduced to adjust the weights of the feature channels, so as to enhance the feature information guiding ability and improve the classification effect of the model. The experimental results show that the classification accuracy of this study on the logo image dataset is better than that of traditional classification methods. This study achieves efficient classification of logo images by incorporating a progressive training strategy with multi-granularity features and a random jigsaw patch generator, which provides a new idea to solve the existing problems in logo image classification.