Global gray-level histogram lacks of spatial information, SIFT feature matching method do not have a high accuracy. In view of the mentioned problems, this paper proposed a method of extracting color moment, and advanced the measuring method of SIFT features. Then combine the two advanced method for image retrieval. The experiment result shows that the algorithm proposed in this paper do a good job in image retrieval.