Dust accumulation is one of the main factors of power loss of photovoltaic modules. In view of the characteristics of dust particles and the high cost of using scanning electron microscopy, this study proposes a scheme to identify dust on photovoltaic panels by using the improved ShuffleNetV2 model. On the basis of the ShuffleNetV2 network model, the Mish activation function is used to integrate the better feature information into the neural network; then the mixed depth convolution is used to ensure the richness of feature extraction. Finally, the coordinate attention mechanism module is used to replace the point-by-point convolution of the tail of the right branch of the basic unit in the ShuffleNetV2 model, so as to improve the accuracy and reduce the calculation amount. The experimental results show that the improved ShuffleNetV2 model has higher accuracy and lower complexity than the existing classical model, which effectively proves that the proposed scheme is feasible.