A dynamic gesture recognition algorithm based on convolutional neural network and support vector machine classification (CNN-Softmax-SVM) is proposed to solve the problems of low recognition rate and few gesture recognition types in monocular vision. Firstly, the fast fingertip detection and tracking algorithm based on YCbCr and HSV color space is employed, which can acquire fingertip trajectory in real time under complex background. Secondly, fingertip trajectory is used as input of joint CNN-Softmax-SVM network, and finally dynamic gesture trajectory is recognized by trained network. The test results show that the combined CNN-Softmax-SVM algorithm can identify the dynamic gesture trajectory well.