During the object tracking, when occlusion occurs, the traditional superpixel tracking algorithm will add the superpixels of the non-target area into the feature space. In the calculation of the candidate sample confidence, the nearest neighbor superpixel in the feature space is used to delimit the cluster attribution of the superpixels in the sample, and the accumulation of the classification error is caused by the excessive number of neighboring superpixels. To solve the problems above, we propose a robust superpixels tracking method. This algorithm uses Bayesian algorithm as the framework. Firstly, we slice the first few frames into superpixels, extract the feature, use the mean shift clustering algorithm and representation model based on superpixel to classify and calculate the class confidence value, and put the feature into feature space. Secondly, the suitable numbers of neighbors can be found with the mean center error of next few frames. Last but not least, during the tracking process, the superpixel is segmented in the specified area of the acquired frame, to extract the feature. The cluster is confirmed with soft classification and the confidence value is calculated. According to the previous frame target position, the Gaussian sampling is collected. We can obtain the sample confidence value with the accumulation of the confidence value. In case of severe occlusion, the sliding window update and the appearance model modification are not carried out, and we continue to use the current model to track. Compared with the traditional tracking algorithm based on nearest superpixel, the algorithm can effectively improve the tracking success rate and reduce the average center errors.