The global image detection of feature points is time-consuming, and the global feature is not of good of stability, which causes the algorithm speed to be slow and the matching accuracy to be low, with the matching effect satisfactory. On the basis of scale invariant feature transform (SIFT) based on the sparse structure of the concept, this study puts forward an image feature matching algorithm based on sparse structure (SSM). It gets the pixel value by sparse sparse degree function, selects pixel highly sparse region, and detects the SIFT feature point of the region, to achieve feature matching by using the best descriptors. Compared with several classical algorithms, the experimental results show that this algorithm has significantly improved in feature matching speed and accuracy, and it can be used for real-time object tracking, image retrieval and image mosaics, and other fields.