According to the characteristics of salt and pepper noise and the defects of the standard median filter algorithm, an improved median filtering algorithm based on support vector machine is proposed in this paper. At first, the algorithm filters the noise image by using median filtering and then defuzzification operations are performed on the filtered image. For those pixels in the defuzzified image whose gray scale values are the maximum or minimum value, support vector machine (SVM) classification is used to decide whether they are noise points. Finally support vector machine (SVM) regression is used to recover original signals from the noise points. Experiments and simulation analyses show that our algorithm can effectively remove salt and pepper noise, and has higher peak signal to noise ratio.