To improve the accuracy and real-time performance of vehicle recognition, this study proposes a vehicle recognition method based on transfer learning. This optimized method improves the accuracy of vehicle recognition, reduces model training time, and improves the robustness of the model by integrating convolutional neural networks and support vector machines. This method first uses a convolutional neural network to train its network on the CIFAR-10 data set. Residual optimization is then applied to a deeper pre-trained network to extract fine-grained features. During the parameter transfer process of the model network, only the pre-trained parameters of the convolutional layer are transferred, and a fully connected layer is added for fine-tuning on the vehicle data set. Finally, the extracted features are directly put into the support vector machine for classification. Detailed model experiments and result analysis demonstrate that this method achieves the highest recognition accuracy of 97.56% and a recognition time of 260 ms per single image, indicating optimized performance in both recognition time and accuracy.