Probabilistic matrix factorization model, making personalized item recommendations according to a user’s historical interaction information, is one of the classic methods in collaborative filtering. Under the assumption of the traditional matrix factorization model, the similarities among different users cannot be used, and prediction is often inaccurate when outliers occur. A clustering-based probabilistic matrix factorization model with category-related conjugate prior distribution is built with user clustering information. Its parameters are regularized by changing the form of the conjugate prior distribution. Through variational inference, the explicit expressions of variational parameters are theoretically derived, and corresponding rating prediction algorithms are thereby established. Both simulation and real datasets show that the prediction performance of the proposed model is better than that of the benchmark model, and it can provide realistic explanations for users’ rating behavior.