A fatigue driving detection algorithm based on deep learning was proposed to solve the problem of poor practicability or low accuracy of existing fatigue driving detection algorithms. First, the Histogram of Oriented Gradient (HOG) feature operator was used to detect the presence of human faces. Secondly, the feature points model was used to realize the face alignment and segment the eye and mouth regions. Finally, the fatigue features of the driver's eyes were extracted by the deep convolutional neural network, and the fatigue features of the driver's mouth were fused to carry out fatigue warning. Experiments show that the proposed method can achieve significant performance improvement in terms of accuracy and real-time performance of fatigue driving detection.