The crop cultivation in China, a populous agricultural nation, plays a vital role in predicting national grain yield and ensuring food security. However, traditional image segmentation techniques are time-consuming and laborious in extracting ground object information from remote sensing images, and may not achieve good results. To solve such problems, this study uses deep learning methods to conduct refined classification and extraction of various crops from high-resolution remote sensing images. In this study, rice, corn, and soybean crops in farmland around Hefei City are taken as experimental samples. In this study, an improved U-Net-based network model for crop extraction is proposed, achieving an extraction accuracy of 94.86%. To further optimize crop boundaries, the CascadePSP edge optimization algorithm is used to refine the experimental results, elevating the accuracy to 96.72%. The experimental results show that the improved U-Net model combined with the CascadePSP algorithm outperforms most deep learning models in extracting crops from high-resolution remote sensing images.