For the loss and loose of CRH train parts, this paper design a set of automatic video image recognition technology for fault detection system. In this important part of image processing, using a template matching algorithm based on contour to detect whether the CRH train parts is lost or loose. This paper elaborates the architecture, implementation of each part of the CRH train fault detection system and puts forward the template matching method based on contour, and gives a detail from the image pre-preprocessing, feature regions to determine the components to the process of setting the minimum score. Application process showed that the application of template matching algorithm to fault detection CRH train parts could achieve the desired results.