The dorsal line of cattle is one of the most important indexes in the identification and classification of Xinjiang brown cattle, and the level of the back line reflects the growth of cattle, which is an important index for selection and breeding. Based on the identification standard of linear type of Xinjiang brown cattle, taking cattle side image (mainly from the chest to the cross step image) as the research object, this paper uses digital image processing method first to process the image binarization, and then to detect the edge of image binarization, realizing the automatic extraction of the edge points of the back line cattle. Finally, through the analysis of the data of the edge of the back line, it gets the automatic classification of the cattle back line, and divides the line into five grades: 45, 35, 25, 15, 5. The higher the score is, the better the growth of cattle is. The experiment shows that the algorithm is effective and feasible, and can get the result of automatic classification of Xinjiang brown cattle back line fast and precisely.