Concrete cracks have negative impacts on the structural load-bearing capacity, durability, and waterproofing. Therefore, early crack detection is of paramount importance. The rapid development of big data and deep learning provides effective methods for intelligent crack detection. To address the issues of imbalanced positive and negative samples, as well as the challenges posed by deep colors and low luminance in crack areas during the crack detection process, this study proposes a crack detection method based on Swin Transformer U-Net (ST-UNet) and target features. This algorithm introduces the CBAM attention mechanism into the network, enabling the network to focus more on the pixel regions in the image that are crucial for crack detection, thereby enhancing the feature representation capability of crack images. The Focal+Dice mixed loss function replaces the single cross-entropy loss function to address the problem of uneven distribution of positive and negative sample images. Additionally, the design of the APSD regularization term optimizes the loss function, addressing the issues of deep colors and low luminance in crack areas and reducing both missed rates and false rates in detection. The results of crack detection show a 22% improvement in IoU and a 17% increase in the Dice index, indicating the effectiveness and feasibility of the algorithm.