Based on the actual application background and the supervised learning situation in which samples of each class comply with Gaussian distribution, we propose a new method for Fisher kernel construction. With the help of classification information in the sample, this method use maximum likelihood estimation rather than EM algorithm to estimate the GMM parameters, which can effectively reduce the time complexity for Fisher kernel construction. A simulation experiment on standard face database shows that the above-mentioned method combined with Fisher kernel classification can not only reduce the time complexity of fisher kernel construction, but also exceed the traditional Gaussian kernel and polynomial kernel in terms of recognition rate. The study of this method will benefit the application of Fisher kernel from speech recognition to image recognition.