To improve the traditional three frame difference of moving target detection method in which the hole edge and false edge tend to appear, this paper puts forward a method based on the fusion of five frame difference and edge detection, moving object detection method. The algorithm firstly uses Canny edge detection algorithm of optimized fast extraction of five consecutive images edge image, and then to the edge of the frame of five consecutive image into five frames difference operation. Finally, by threshold segmentation and morphological processing is completed on the extraction of target. The method is simple in calculation and the experimental results show that the algorithm has high accuracy, good connectivity, and can meet the requirements of real-time detection.