The automatic production of wood-based panel has been realized with the development of continuous press production line, but the defect detection is still manual. As an important part of detection, defect recognition is a process of using a classifier to identify defects based on feature value. For the reason of the continuous production of wood-based panels, the defects need to be identified quickly and accurately. Therefore, a cart tree is proposed to identify the defects of the wood-based panel in this study. The defect features of shape and texture are firstly obtained using image preprocessing and image segmentation, and then the cart tree is generated by Gini exponent, at last defects are identified by using the cart tree. But it is easy to cause the problem of overfitting using cart tree without pruning, so the study obtains the optimal subtree by using the cost complexity algorithm and 10 cross-validations. The experimental results reflect that the accuracy rate of defect recognition reaches 93% with the proposed cart tree, which can satisfy the requirements of real-time and accuracy on defect identification.