In order to quickly drive accident vehicles away from the scene and ensure a clear road during a minor traffic accident, this study proposes a vehicle collision detection and liability determination model. First, the study combines the SSD (single shot multibox detector) target detection algorithm and the MobileNet lightweight deep network model to make improvements and obtain the position and size information of the moving target in each frame of video images, so as to identify and detect the vehicle. Secondly, the study employs a Kalman filter to establish a corresponding matching relationship between moving targets in consecutive image frames, predict their motion states, and judge their positions and motion trend, in a bid to track the vehicle. Then, the study determines whether there is a collision by the intersection over union of the vehicle target detection frame. Finally, according to the speed and direction information of the vehicle on a straight road, the liability of the accident vehicle is determined under the road safety regulations and the fast method of motor vehicle accidents. The results show that the research can help to detect and determine the liability during vehicle collisions caused by rear-end collisions and lane changes on straight roads.