In this paper, a new algorithm of taillight signals detection and recognition based on image is proposed. Firstly, the algorithm detects and tracks vehicle taillights using color and symmetry features of the images. Secondly, we employ parameters-optimized Least Squares Support Vector Machines(LS-SVM) to classify the current states of detected taillights. Finally, we set up state chains from past frames to recognize the signals of taillights. Through the experiments on the real road, we can see that the recognized signals can correctly indicate the behaviors of brake and turn of preceding cars, which shows that our algorithm is valid.