The color features, as an important visual information of image, have been widely used in the Content-Based Image Retrieval(CBIR) because of the simple calculation and better robustness. According to the color features of the image, this paper makes systematic discussion and research on color space(color space transformation), color quantification(HSV color space), color features extraction and color characteristic similarity measurement. Then, through classic color segment histogram algorithm, the entire process of Color-based Image Retrieval is proposed. Finally, the existent problems and further research direction of the color-based image retrieval is summarized.