As a key step in binocular 3D reconstruction, the binocular stereo matching algorithm completes the transformation from planar vision to stereo vision. But how to balance the running speed and accuracy of the binocular stereo matching algorithm is still a difficult problem. In this study, focused on that existing local stereo matching algorithm has low matching accuracy in specific regions such as weak texture and depth discontinuity, while considered the real-time performance of the algorithm at the same time, an improved stereo matching algorithm based on cross-scale guided filtering is proposed. Firstly, the two cost calculation methods of SAD and Census transform are combined, and then the cost aggregation is performed by using cross-scale guided filtering. When calculating the disparity calculation, a judgment criterion is used to judge the reliance of the disparity value corresponding to the minimum aggregation cost of each pixel in the image. When it is judged that the corresponding disparity value is unreliable, an adaptive window based on gradient similarity is constructed for the pixel, and the disparity value corresponding to the pixel is corrected based on the adaptive window. Finally, the final disparity map is obtained by parallax refinement. Experimental results on standard stereo image pairs on the Middlebury test platform show higher accuracy than traditional guided filter based stereo matching algorithms.