In view of the vehicle's abnormal behaviors in the artificial monitoring, such as speeding, illegal lane changing and red light running, this study proposes a method for detecting abnormal behaviors of vehicles based on video analysis technology. First, it uses ViBe (Visual Background Extractor) method to get the foreground image. It tracks the corners by using the Lucas-Kanada optical flow method, getting the corners velocity and direction information. Then, it uses the mean shift method to cluster the two motion features to get the statistical histogram after clustering. Finally, it judges the abnormal behavior of the vehicle with the Euclidean distance of the motion characteristic entropy and the two motion characteristic scalars to the cluster center. The experimental result shows that the two methods can detect vehicle's abnormal behaviors accurately and in real time.