This paper gives a method of realizing a video moving vehicle detecting system based on DM6446 processor, and describes the system’s hardware architecture and software design ideas. The methods of ARM-side and DSP-side communication and double-buffer switch are given combining the characteristic of dual-core of DM6446 processor. Background modeling is made on the traffic video data from DM6446 via difference accumulation. Background subtraction is used to detect vehicle movement areas. With the adoption of other algorithms such as Otsu, morphological filtering and region growing, the system is finally completed on this hardware platform. Experimental results show that the system is good in moving vehicle detection.