The vehicle detection system in current use widely uses host to centrally process images passed back to the server, and it thus has shortcomings, such as large output data, long processing time and so on. This paper proposes a constructing plan using embedded hardware platform according to the composition of ITS system. The video image is emphatically analyzed, and the tracking algorithm is also optimized. The basic image processing tool OpenCV is used to select the preferable edge algorithm by comparing the canny operator and sobel operator. Considering smearing, the background is extracted with the improved averaging method (take interval frames). Finally the nuclear track is chosen for dynamic target tracking, and it is applied in the system for getting the stability of dynamic target tracking.