According to the X-ray weld image collected by a steel pipe factory and the study on diversity and morphological variability of weld defects, a weld defect identification algorithm based on rotation invariant HOG feature extraction is proposed. First of all, we classify different types of defects detected to extract ROI of each image, all of which constitute the defect samples required by the experiment. By means of scale transformation and circular cell division, we obtain HOG characteristics with scale invariance and rotation invariance. Then all the sample features are reduced by PCA dimensionality reduction. The dimension is determined by the contribution. Finally, the LSSVM model is used to identify the defects. By studying the effect of block overlap on the recognition accuracy rate, it is found that the higher overlap range, the higher correctness in a certain unit. The algorithm improves the accuracy of defect recognition by improving the traditional HOG feature extraction method.