To improve the pedestrian detection performance, this study proposes a pedestrian detection algorithm based on improved YOLOv4 by combining SqueezeNet, attention mechanism, dilated convolution and Inception structure. An attention module named D-CBAM is proposed which is combined with dilated convolution. It is introduced to the feature enhancement part to select useful information from the extracted features. The residual connection is also used in this part to enhance feature reusability. In addition, an Inception-fire module is proposed by combining the “squeeze-expand” structure of SqueezeNet and the multi-scale convolution kernel structure of Inception, which replaces the continuous convolution layer in the network. Increasing the width of the network improves the performance of the algorithm and reduces network parameters. According to the characteristics of pedestrian detection and focal loss, the loss function is improved. The detection ability is enhanced through the addition of weights to the positive and negative samples and the hard and easy samples respectively and the strengthening of the training on positive samples and hard samples. The detection accuracy of the improved YOLO algorithm on INRIA person data set can reach 94.95%, which is 4.25% higher than that of YOLOv4. The parameters of the model are reduced by 36.35%, and the detection speed is improved by 13.54%. In short, the improved algorithm shows better performance in pedestrian detection than YOLOv4.