Based on the rapid development of intelligent transportation, this work studies the lane detection and vehicle tracking technology of high-speed sections. For multi-lane detection, the road surface is segmented by using the feature that the gray level difference between the road surface and the dividing line is rather large. Then, the line equation and the Catmull-Rom Spline interpolation algorithm are used to fit the lane dividing line. For single-lane detection, the single lane is first effectively segmented based on the HSV color space and Sobel edge extraction method, and then the lane separation coordinate points are extracted in the perspective transformation space and the segmentation line is fitted with a quadratic polynomial. Aiming at the vehicle detection, the HOG+Gentle-Adaboost classification algorithm is firstly used to detect the vehicle in front of the unmanned vehicle, and then the shadow of the vehicle is detected based on the characteristics of the shadow at the bottom to verify the authenticity of the vehicle area detected by the learning algorithm. For vehicle tracking, the dynamic second-order autoregressive model method is used to predict the state of the vehicle. For the inherent particle degradation problem of particle filtering, this study innovatively introduces the Thompson-Taylor algorithm to improve the defects of particle degradation and low diversity. The lane detection and vehicle tracking algorithms in this study can be easily transplanted on the embedded platform with high reliability and accuracy, and further to realize the lane departure warning and forward collision avoidance system.