Medical images are helpful for the diagnosis, treatment, and evaluation of diseases. Accurate segmentation of organs in medical images is of great practical significance to assist doctors in diagnosis. Due to the low contrast between organ parts and surrounding tissues in medical images, the edges and shapes of different organs are very different, which increases the difficulty of segmentation. To solve these problems, this study proposes a semantic segmentation network for medical images based on a convolutional neural network and Transformer, which effectively improves the accuracy of semantic segmentation of medical images. The feature extraction part uses a ResNet-50 network structure, and a Transformer module is employed to expand the receptive field after feature extraction. In the process of up-sampling, multiple skip connection layers are added, and the feature extraction information of each stage is fully utilized to make the resolution close to that of input images. The experimental results on the segmentation dataset of gastrointestinal medical images prove that the proposed method can effectively segment organs and tissues in medical images and improve the segmentation accuracy.