Video object tracking technology has great applied value in the fields of traffic and military. In this paper, a video matching object tracking algorithm, which is comentropy-based, is proposed. First of all, the concept of comentropy is introduced. Secondly, the feature of the video object is described by comentropy, and the algorithm which combined with mean-shift is designed with feature match in two different feature spaces, RGB and HSV. Finally, the experiment simulation is presented. The experiment results show that the proposed algorithm can track the object steady and real-time. This algorithm can achieve the desired effects and it is a practical algorithm.