For the parallel edge points fitting in image measurement, this paper combines with the least square method to classify parallel edge points fitting through clustering analysis algorithm to eliminate the noise in the edge points image. The traditional algorithm based on image characteristics dividing area artificially to fitting respectively is solved. The result achieves the purpose of automatic image processing by machine.