Aiming at the problem that hand gesture recognition system based on ordinary camera is susceptible to the different lighting conditions and complex background, a fingertip detection and hand gesture recognition algorithm based on Kinect depth image is proposed. First, we get depth image by Kinect sensor. Then the hand region is extracted by putting the depth of thresholds on hand point detected by using OpenNI library. Fingertip detection based on convex hull and curvature is proposed. After the number of fingertips and the location of fingertips being detected, it calculates a feature vector including the number of fingers, the angles between fingertips and horizontal of the hand, the angles between two consecutive fingers, and the distance between fingertips and hand center point. Finally, a support vector machine(SVM) is applied to identify nine scheduled number hand gesture. Five experimenters are invited to perform 9 different hand gestures in the complex environment, which each gesture is repeated at thirty times and the angle of hand gesture is different every time. The experiment results show that this algorithm can detect the number and location of fingertips, and the recognition rate of nine hand gesture is 97.1% on average. This proposed method uses simple features and has good robustness, also it is real-time.