In this paper, a moving object detection system based on FPGA is designed. It uses the modular design mode and pipeline processing method, and makes full use of the FPGA features of high-speed parallel processing and DDR2 SDRAM large capacity characteristics. The system uses the combined methods of the background difference method with single Gauss background modeling and frame difference algorithm to achieve the detection of moving objects. Practical test results show that the target detection system can effectively achieve the detection of moving objects and meet the needs of real-time, and has good performance.