This study proposes a multi-level knee cartilage image segmentation network to address the challenge of limited annotated data in knee cartilage MRI segmentation. This network incorporates a medical attention mechanism and integrates conditional random fields to create a hierarchical attention framework. By separating the data stream into global and local streams, the network captures both global features and local details of the image, thus enhancing segmentation accuracy. In addition, an axial attention mechanism is introduced to reduce computational load, effectively simplifying the calculation process and minimizing model parameters. The integration of hierarchical segmentation strategies and conditional random fields enables a deeper exploration of the interdependence between categories, improving the network’s ability to capture key features. The proposed method’s effectiveness is validated through experiments on two public datasets, K-Space and MOST. Experimental results demonstrate that even with limited data annotation, the method achieves high-precision knee cartilage image segmentation. Compared to current state-of-the-art methods, the proposed approach shows significant improvements in evaluation metrics such as the Dice similarity coefficient (DSC) and 95% Hausdorff distance (95HD).