Traditional video image data processing usually requires software, but there are some problems, such as heavy workload and long storage time. To solve this problem, this paper proposes a hardware process configuration based on FPGA, to improve the processing speed of the system and real-time. For the image processing problems, it uses the background subtraction to detect moving targets and remove the static scenes from images by Binarization, then does morphological filtering to eliminate voids and noise points, finally, extracts the target center to achieve the purpose of detecting moving targets. It joins sobel edge detection module at the same time, extends moving target detection architecture. According to the shape of the object and its path of movement, system can identify objects and categorize their test results when detects the target. In addition, for some specific environment, it sets the appropriate restricted area, when the center of the target is into the restricted area, makes the appropriate judgment. After the hardware design and Verilog HDL program is completed, doing simulation in Modelsim can get the right target detection results.