In skin lesion image segmentation tasks, U-Net suffers from insufficient multi-scale adaptability, inefficient cross-layer feature fusion, and computational redundancy that leads to edge information loss when processing dermoscopy images. This study proposes a hierarchical pyramid attention network (HPANet) that achieves dual optimization of multi-scale feature capture and cross-layer feature transmission through pyramid attention module and dual-path feature fusion mechanism. The dual-path adaptive fusion module combines CNN and Transformer dual-branch features, enhancing information interaction of complementary features through channel attention and compressed spatial attention, while utilizing bilinear interaction and residual connections to alleviate feature dilution problems. The pyramid attention module integrates hierarchical multi-kernel convolution, depthwise separable downsampling, and patch-wise spatial-channel attention mechanisms to significantly improve multi-scale lesion feature capture capability. Experimental results demonstrate that the proposed architecture outperforms mainstream models on both ISIC 2017 and ISIC 2018 datasets, confirming its dual advantages in lesion boundary preservation and small lesion detection.