To study the influencing factors on the skid resistance of asphalt pavements and accurately predict the skid resistance of the pavements, this study resorts to the Gocator 3110 three-dimensional (3D) intelligent sensor to obtain the surface textures of asphalt mixture specimens and employs the pendulum friction tester to measure the surface friction coefficient of the specimens. Regarding the abnormal data in the 3D point cloud data of the textures, the study proposes a radial basis function (RBF)-based neighborhood interpolation algorithm to improve data quality. Then, the feature parameters of 10 typical macro-textures are calculated with the reconstructed 3D point cloud data of the textures, and the redundancy factor is removed by Pearson coefficient correlation analysis to improve the input features of the model. A prediction model for the skid resistance of asphalt pavements integrating an improved gray wolf optimization (IGWO) algorithm and XGBoost is developed to predict the friction coefficient of asphalt pavements. The results show that the prediction accuracy of the proposed model is better than that of the multiple linear regression model, the support vector machine regression model, and the XGBoost model based on grid search.