Aiming at the problem that traditional machine learning methods are not ideal in terms of effect and time for identifying crop leaf pests and diseases with small samples and multiple categories, this study utilizes an improved ResNet model to realize the recognition of crop pests and diseases. By adding dropout layers, activation function, maximum pooling layer, and attention mechanism, the robustness and feature capturing ability of the model is improved, and the accuracy of pest and disease recognition with a lower number of model parameters is achieved. Firstly, the images obtained from the public dataset Plant Village are preprocessed and enhanced, and the ReLU activation function is replaced by PReLU to solve the problem of neuron necrosis in the part of the ReLU function less than 0. Then, a dropout layer is added before the global average pooling layer, and a reasonable threshold value is set to effectively avoid the occurrence of overfitting and to enhance the robustness of the model. In addition, a maximum pooling layer is added between the dropout and global average pooling layer, which not only expands the receptive field of neurons, but also helps the model to obtain the most significant features of local pests and diseases, reduce the noise effect from image background, and realize secondary feature extraction. Finally, the CBAM attention mechanism is embedded, which makes the model automatically learn the most important channel information in the input feature maps and weight it between the channel and space to better capture the semantic information in the images. Experimental results show that the improved model recognizes the test set with an accuracy of 99.15% with a model parameter count of only 9.13M, which exceeds the accuracy of Xception, InceptionV3, and the original ResNet by 1.01, 0.68, and 0.59 percentage points, respectively, and reduces the model parameter count. This provides a state-of-the-art crop disease recognition deep learning method.