Considering the strict requirements for data and limited application for the traditional method of the object extraction from Remotely Sensed Imagery (RSI), a building extraction algorithm based on the nonlinear scale-space filtering is proposed. Firstly, nonlinear scale-space of each band in the multispectral RSI is constructed, and the iterative filtering is done. Then, the first valley point in standard deviation curve of the global image is searched to stop the iteration process. Finally, the binarization of filtering results using the Otsu method for each band is made. To verify the validity of the proposed method, an aerial image covering Fuzhou, China is chosen to test and compare with the similar method. Experimental results show that the proposed method can smooth the noise while preserving the building edges information, and has better effect for extraction of the closely spaced buildings. Moreover, the recall of the proposed method increases more than 5% in the premise of ensuring the precision.