In endoscopic gastric polyp detection based on computer vision, efficiently extracting the features of images of small polyps is a difficulty in the design of a deep learning-based computer vision model. To solve this problem, this study proposes a detection model based on an improved you only look once version 4 (YOLOv4), namely YOLOv4-polyp. Specifically, on the basis of YOLOv4, this study adds a convolutional block attention module (CBAM) to enhance the feature extraction capability of the model in complex environments. Then, a lightweight CSPDarknet-49 network model is designed to both reduce the complexity of the model and improve its detection accuracy and detection speed. Finally, considering the characteristics of the gastric polyp datasets, the K-means++ clustering algorithm is used for the cluster analysis of the gastric polyp datasets and the attainment of the optimized anchor box. The experimental comparison results show that compared with the classical YOLOv4 model, the proposed YOLOv4-polyp achieves favorable detection performance on the two datasets as it improves the average detection accuracy by 5.21% and 2.05%, respectively, without compromising the detection speed.