Retinal vessel segmentation is vital for assisting doctors in diagnosing ophthalmic diseases, including diabetic retinopathy, macular atrophy, and glaucoma. The attention mechanism is widely used in U-Net and its variants to improve the vessel segmentation performance. For more accurate retinal vessel segmentation and exploration of high-order and global context information, we propose a multi-scale high-order attention network (MHA-Net). The multi-scale high-order attention (MHA) module first extracts multi-scale and global features from the high-level feature maps to compute the initial attention map, enabling the model to handle medical image segmentation with variable scales. Then the high-order attention constructs the attention map through graph transduction followed by the extraction of high-level features at high order. We further embed the proposed MHA module into an efficient encoder-decoder structure for retinal vessel segmentation. Comprehensive experiments are conducted on the color fundus image dataset DRIVE, which indicates that the proposed method improves the accuracy of retinal vessel segmentation effectively.