The driver assistance system is considered the first choice for solving traffic safety problems. The basis of developing a driver assistance system is to accurately recognize the vehicle behavior for applications in aspects such as vehicle safety warning, path planning, and intelligent navigation. The existing behavior recognition methods based on the support vector machine model, hidden Markov model, and convolutional neural network still face the imbalance problem between calculation amount and accuracy. This study proposes a Gaussian mixture hidden Markov model, which is a combination of the hidden Markov model and the Gaussian mixture model. The model is experimentally verified on the NGSIM data set from the Federal Highway Administration of the USA, and the results reveal that the model has higher accuracy in the recognition of free lane-changing behavior. Additionally, this study optimizes the parameters of the proposed model to achieve the best recognition effect and provide a reference for the vehicle behavior recognition of intelligent driving in the future.