The adaptive median filtering algorithm can effectively filter the impulse noise of image, however, with the noise density increasing, its filtering performance decreases progressively. For the improved median filtering algorithms of current, there are also relevant limitations. Against the limitations of the median filtering algorithm, an improved adaptive median filtering algorithm is proposed. It does noise detection based on the gray extremum of the filtering window. And it replaces the noise point with the gray median of the filtering window. If the gray median is noise point, it increases adaptively the filtering window to take a new gray median. If the filtering window has increased to the maximum size of allowed, and the gray median is still noise point, it takes the gray mean of the pixels except the gray extremum in the filtering window. Simulation experiment has been carried out for standard image and medical image, the results and datum of the filtering experiment demonstrate that, with the noise density increasing, the filtering performance of the standard adaptive median filtering algorithm decreases progressively; and the filtering performance of improved adaptive median filtering algorithm is still good, it maintains well the edges and details of image while filtering effectively the noise.