As the inaccurate estimation of atmospheric light curtain and atmospheric light result in the halo effect, color distortion, and low contrast in the process of haze image restoration, a dehazing algorithm based on weighted least square (WLS) filtering and restorative controlling factor is proposed. Firstly, this study analyzes the principle and performance of the WLS filter, which can be utilized to effectively estimate the atmospheric light curtain. Secondly, with the assistance of the Sobel operator, the binary image edges are detected. The number of edges and the mean value of pixels are taken as the bases of the quad-tree space index, which improves the estimation accuracy of the atmospheric light. Finally, according to the causes of color distortion in the sky area, a restorative controlling factor is introduced to improve visual effects. Experimental results show that the mean gradient obtained by this method increases by 58.03%, and the information entropy increases by 2.88%. In particular, the running time relatively decreases by more than 50%. The proposed method achieves better restoration in terms of contrast, visibility, and color fidelity of the haze image containing complicated near and distant scenes that mixed dense haze, mist, and sky area.