In the field of image segmentation and identification, the existing deep learning methods mostly perform tasks by high-precision semantic segmentation methods, which lead to a slow network inference speed, large amount of calculation, and difficult actual application. A real-time network model with better performance, namely BiSeNetV1 is used, and the extended spatial path convolution structure, spatial pyramid attention mechanism (SPARM), simplified iterative attention feature fusion (S-iAFF) module, and other optimization strategies are applied. As a result, a real-time BiSeNet_SPARM_S-iAFF network is designed for rock debris image segmentation. The extended spatial path convolution structure can obtain more abundant spatial features of rock debris images. The context path uses the optimized SPARM to further refine high-level semantic feature extraction. Finally, S-iAFF is used to enhance the fusion degree between low-level spatial and high-level semantic features in the feature fusion stage. The experimental results indicate that the mean intersection over union (mIoU) of the BiSeNet_SPARM_S-iAFF network on the RockCuttings_Oil dataset is 64.91%, which is 2.68% higher than that of the BiSeNetV1 network, and the precision of the improved network is close to that of the most high-precision semantic segmentation methods, while the number of parameters is greatly reduced, and the inference speed is significantly improved.