An image set consists of a large number of different images, and these images represent the same person.In the real life, the dates of image sets are nonlinear due to the variation from viewpoint, emotion and illumination.Inspired by the manifold theory knowledge, we assume that modeling each image set as a manifold will be more efficient compared with the traditional method of modeling an image set as a subspace intrinsically.Because the images in an image set are different from each other, it is unreasonable to model an image set as a linear subspace without ignoring the data structure in the set, which may decrease the recognition rate ultimately.In the paper, we introduce a method of improved multi-manifold based face recognition for image sets, and propose a new method for computing the distance between two subspaces.In addition, in order to get the minimal manifold distance, we choose the mean value of closest pair of subspaces as the manifold distance.We call this new method, an improved multi-manifold (IMM).Experimental results on the public available face databases, CMU PIE, demonstrate that the new method outperforms the competing methods