Considering strong noise interference and difficult shadow detection in high-resolution remote sensing images of high-rise buildings, this study proposes a shadow detection method for remote sensing images of high-rise buildings, which is based on the combination of improved threshold segmentation and residual attention networks. Firstly, a threshold segmentation model is built by the improved maximum inter-class and minimum intra-class threshold segmentation algorithm, and on the basis of the connected domain characteristics and end-point positional constraint relationships between contours, the Euclidean metric algorithm is used to repair the broken contours for the shadow contours. Then, the generative adversarial network (GAN) model is used to expand the misjudgment data set. Finally, the residual network is improved, and the attention mechanism is added to the feature map for global feature fusion. In different scenes, the proposed method is compared with the radiation model, histogram threshold segmentation, color model-based shadow detection method, support vector machine (SVM), visual geometry group (VGG) network, Inception, and classification network of residual networks, and the proposed method has a comprehensive misjudgment rate and missed detection rate of 2.1% and 1.5%, respectively. The results reveal that the proposed algorithm can better complete the segmentation and detection of shadow areas, which is conducive to saving human and material resources and assisting staff with their work such as interpreting remote sensing information and establishing remote sensing archives. The proposed method has practical value.