In view of the problem of incomplete feature extraction and large model size caused by ignoring global features in shallow networks in existing pneumonia medical image recognition research, a lightweight model based on convolutional neural network (CNN) and attention mechanism is proposed to improve the recognition efficiency of pneumonia types. A lightweight model structure is used to reduce the number of model parameters. By increasing the convolution kernel, efficient channel attention and self-attention mechanisms are introduced to solve the problem of loss of important network information and the inability to extract underlying global information. Local and global information is extracted in parallel through dual branches, and multi-scale channel attention is utilized to improve the fusion quality of the two. The CLAHE algorithm is employed to optimize the original data. The experimental results show that the accuracy, sensitivity, and specificity of the model are increased by 2.59%, 3.1%, and 1.38% respectively compared with those of the original model while ensuring lightness, and the proposed model outperforms other current excellent classification models and has stronger practicability.