Given the low recognition rate and the difficulty in detecting small cracks in the asphalt pavement under complex background, the crack detection method based on improved Faster-RCNN is proposed. First, the road surface images are collected by the multifunctional road detection vehicle, and 13 000 pictures are divided into training sets and test sets at a ratio of 8:2. Then VGG16, MobileNet-V2, and ResNet50 networks are utilized to replace the feature extraction network in the Faster-RCNN model to identify the cracks. The results show that the combination of ResNet50 and Faster-RCNN can achieve the best result with an accuracy of 0.805 8. The cracks are distributed on the same level without hierarchical information. Therefore, other ResNet networks are expected to work better with the Faster-RCNN model. However, it turns out that ResNet50 still outperforms ResNet18 and ResNet101. In the case of missed detection of small cracks, the convolutional block attention module (CBAM) module is also introduced into ResNet50 and the influence of different insertion positions on the detection accuracy is compared. Experiments show that the improved Faster-RCNN model has a detection accuracy of 85.64%, which can effectively detect small cracks under complex backgrounds.