According to the continuous tracking of multi-camera to multi-objective under the broad condition for different scene, this paper proposes a matching algorithm based on characteristics of the pure goal of robust adaptive SIFT (Scale Invariant Feature Transform). This method establishes a CamShift (Continuously Adaptive Mean Shift) tracking device for each pure goal which withdraws from the video image. It uses adaptive criterion space factor to get detail characteristic of goal. It uses the bilateral matching strategy based on BBF (Best Bin First) to elimination the error matching points. When the quantity of the goal key points is too few to satisfy the calculation the precise key point position of computation three dimensional quadratic function, it designs the adaptive criterion Harris vertex examination law to supplement the new spot. The experiment of continuous tracking outdoor vehicles in different settings indicates that this algorithm timeliness is good and its adaptive ability is strong. Compared with other algorithms, this algorithm consumes less match time, but is high in tracking precision.