The small-sized traffic signs actually detected by the YOLOv2 algorithm are of poor quality, low recognition rate, and poor real-time performance. This study proposes a traffic sign detection method based on improved YOLOv2. Firstly, the image is enhanced by histogram equalization and BM3D method, with high-quality images. Moreover, the top-level convolutional layer output feature map of the network is finely divided to obtain fine-grained feature maps to detect high-quality, small-sized traffic signs. Finally, the loss function is improved by normalization and optimization of the confidence score ratio method. Experiments were carried out on a new data set combining CCTSD (China Traffic Sign Detection Dataset) and TT100K dataset. Compared with the YOLOv2 network model, the network recognition rate increases by 8.7% and the recognition speed of the model is improved by 15 FPS. Experimental results show that small-sized traffic signs can be accurately detected by proposed method.