Abnormal behaviors of vehicles may cause traffic accidents or even economic losses and casualties. Accurate recognition of abnormal vehicle behaviors can prevent potential hazards. To tackle the problems in existing studies, such as difficulty to retain the time characteristics of data, this study proposes a recognition model of long short-term memory (LSTM) neural network with an attention layer, and trains and verifies the proposed model by using abnormal vehicle trajectories in real traffic scenes. The experimental results show that the proposed model can effectively recognize abnormal driving behaviors with accuracy reaching 98.4%.