The positioning of finger knuckle in hand image is the basis for the hand gesture recognition and motion tracking in intelligent assembly by human-computer interaction with machine vision. The accuracy of hand knuckle information has direct influence on the gesture description and behavior recognition. Considering the random distribution characteristics of knuckle images, the image preprocessing has made by homomorphic filtering to enhance the image features. Classification feature learning of clustering set on knuckle image is finished based on the Gaussian process model. The characteristics model of the data distribution is learned with the clustering measure of the sample. The two type feature model from the feature learning is a detector for image feature, and the detecting results are the two likelihood values of the image. The finger knuckle target is recognized directly according to the estimation results while the two types model likelihood value as the input value which is marked by the positive and negative samples. The hand knuckle recognizing experiments with different location are held, and the knuckle detection is carried out with our own created test knuckle library. It shows from the experimental results that the posterior probability distribution can be obtained directly, and the recognition accuracy and efficiency of target are improved. The algorithm presented above is feasible.