Aiming at the small target scale and low detection accuracy in traffic signal detection, this study proposes a traffic signal detection algorithm based on improved YOLOv5s. Firstly, a feature pyramid module RSN-BiFPN is constructed to fully integrate traffic signal features of different scales to reduce target missed detection and false detection. Secondly, a new feature fusion layer and prediction head are introduced to improve the perception performance of the network for small objects and enhance detection accuracy. Finally, the EIoU function is adopted to optimize the loss and accelerate network convergence. Experiments conducted on the public dataset S2TLD show that compared with the basic network, the precision rate of the proposed method is increased by 4.1% at 96.1%, the recall rate is 95.9% with an increase of 3%, and the average precision is increased by 1.9%, reaching 96.5%. Meanwhile, the improved algorithm achieves a faster detection speed of 22.7 frames per second. The proposed method can realize rapid and accurate detection of traffic lights and can be widely employed in the research on analyzing traffic lights.