To address issues of the limited receptive field and insufficient global information of the U-Net model in MRI brain tumor segmentation, this study?proposes an improved U-Net model, i.e., PyCSAU-Net, by introducing non-local self-attention mechanism and multi-scale pyramidal convolution. The given model leverages the three-dimensional U-Net as the baseline and introduces the extended three-dimensional non-local attention to the horizontal connection of the fourth layer, which solves the issue of insufficient long-term modeling ability caused by the limited convolution kernel size to a certain extent, thus improving the segmentation performance. Moreover, it replaces the normal convolution by three-dimensional pyramidal convolution with multi-scale characteristics to capture more discriminant deep features of brain tumors at multi-levels and multi-resolutions. The segmentation results of 0.904/0.901, 0.781/0.774, and 0.825/0.824 are achieved on the publicly BraTS 2019 and BraTS 2020 validation datasets on the whole tumor, enhanced tumor, and tumor core, respectively. It demonstrates the effectiveness and competitiveness of PyCSAU-Net for the brain tumor segmentation task.