DeepLabV3+ ignores the loss of part of detail information due to the importance of features at different scales in the feature extraction stage, which results in imprecise image segmentation. In response, this study proposes an improved algorithm integrating dual-branch feature extraction and attention mechanism. The feature map extracted by the ResNet101 backbone network is used as the input feature of the attention mechanism, which solves the problems of network degradation and gradient disappearance and also captures the image details ignored by DeepLabV3+. The dual-branch feature extraction mechanism expands the feature extraction capability and refines the image edge information to optimize the uneven attention of the network to features at different scales. At the same time, the CE loss function and the Dice loss function are jointly used to reduce the influence of background by focusing on foreground samples and improve segmentation accuracy. The experimental results show that the mean intersection over union (MIoU) of the improved algorithm on the PASCAL VOC 2012 and CityScapes datasets reaches 79.92% and 68.59%, respectively. Compared with the classical algorithm and other improved algorithms based on DeepLabV3+, the proposed algorithm obtains a better segmentation effect.