Non-intrusive load decomposition is an important part of the intelligent power consumption system, which can deeply analyze the power consumption information of users and is of great significance to load forecasting, demand side management, and power grid security. This study proposes a non-intrusive load decomposition method based on the improved particle swarm optimization factorial hidden Markov model (IPSO-FHMM). Gaussian mixture model (GMM) is used to cluster the states of individual loads. The total load model is represented by an FHMM. Since the Baum-Welch algorithm tends to converge to the local extremum, the PSO algorithm with linearly decreasing weights is introduced into the parameter training of FHMM. Simulation experiments using the AMPds2 dataset show that the model can effectively improve the decomposition accuracy.