In recent years, the urban public security poses a new problem to the sustainable development of social economy. Therefore, how to effectively monitor the abnormal situation of the crowd has become a hot issue. Due to the large number of moving targets and the changing of the crowd, it is difficult to study the abnormalities of the crowd by tracking the moving objects. The study shows that when the crowd is abnormal, the most obvious change is the movement speed of the crowd and the direction of the movement will suddenly change. For example, from static state or slow walking to fast running, and the sudden change of the motion direction. The corresponding motion vector of the video frame will undergo the same change. Thus, we propose the fast detection algorithm for the crowd based on motion vector. The experimental results show that the algorithm proposed can detect the abnormal movement of human beings in real time and effectively.