The traditional hand contour feature extraction can not deal with the effect of facial skin color, occlusion, lighting in the flight simulation environment. The traditional Fourier descriptor features are susceptible to background and hand posture changes, and have limited ability to describe gestures, etc. Hence, this study proposes a methed to improve the traditional hand segmentation and feature extraction methods. Firstly, skin color processing is performed on the collected data set, and then the 22 key points of the hand are detected in combination with the called hand key point model, and an eight-way seed filling algorithm is used for image segmentation. Then, the contours of the hand and key points are connected to the skeleton to extract the features of the Fourier descriptor algorithm. Finally, the Support Vector Machine (SVM) algorithm is used to train and recognize the extracted gesture feature data set. The experimental results show that the method in this study has good hand segmentation, feature extraction is not easily affected by changes in the background and hand posture. Hence, it can well cope with interference in a complex background in a flight simulation environment, with the recognition accuracy reaching 98%. The research presented in this paper has a certain role in improving the traditional gesture recognition algorithm and has very important application value in the field of hand interaction technology.