Lane detection is one of the most important modules in self-driving tasks. Lane detection is a challenging task as the structure of the lane line is special, and the detection is easily affected by various environments (such as lighting transformation, obstruction, and the blur of the lane line). Considering the traditional Convolutional Neural Network (CNN) is unable to learn fine spatial features of the lane line directly, in this study, the spatial feature aggregation module is employed to enhance the features extracted by CNN in spatial dimensions, providing rich spatial features for the cascade lane predictor. The experiments show that the module learns fine global information by aggregating feature maps in horizontal and vertical directions and thus improves the performance of the lane detection algorithm in different environments without reducing the detection speed.