A large number of research behavioral methods are focused on detecting simple actions such as walking, jogging, or jumping, while less research is on violence or aggressive behavior, but these studies are useful in some surveillance scenarios, such as: Prison, self-help banks, shopping malls and so on. Traditional methods of violent behavior recognition research mainly use a priori knowledge to manually design features. In this paper a violence detection method based on 3D-CNN structure is proposed. The three-dimensional deep neural network directly manipulates on the input, which can be a good extraction of violent behavior of time and space characteristics of information. It can be seen from the experimental results that this method can identify the violent behavior better than the characteristics of hand-craft features.