Accurate estimation of tropical cyclone intensity is the basis of effective intensity prediction and is crucial for disaster forecasting. Current tropical cyclone intensity estimation technology based on deep learning shows superior performance, but there is still a problem of insufficient physical information fusion. Therefore, based on the deep learning framework, this study proposes a physical factor fusion for tropical cyclone intensity estimation model (PF-TCIE) to estimate the intensity of tropical cyclones in the northwest Pacific. PF-TCIE consists of a multi-channel satellite cloud image learning branch and a physical information extraction branch. The multi-channel satellite cloud image learning branch is used to extract tropical cyclone cloud system features, and the physical information extraction branch is used to extract physical factor features to constrain the learning of cloud system features. The data used in this article include Himawari-8 satellite data and ERA-5 reanalysis data. Experimental results show that after introducing multiple channels, the root mean squared error (RMSE) of the model is reduced by 3.7% compared with a single channel. At the same time, the introduction of physical information further reduces the model error by 8.5%. The RMSE of PF-TCIE finally reaches 4.83 m/s, which is better than most deep learning methods.