Abstract:Knowledge distillation is widely adopted in semantic segmentation to reduce the computation cost. The previous knowledge distillation methods for semantic segmentation focus on pixel-wise feature alignment and intra-class feature variation distillation, neglecting to transfer the knowledge of the inter-class distance, which is important for semantic segmentation. To address this issue, this study proposes an inter-class distance distillation (IDD) method to transfer the inter-class distance in the feature space from the teacher network to the student network. Furthermore, since semantic segmentation is a position-dependent task, thus this study exploits a position information distillation module to help the student network encode more position information. Extensive experiments on three popular semantic segmentation datasets: Cityscapes, Pascal VOC, and ADE20K show that the proposed method is helpful to improve the accuracy of semantic segmentation models and achieves great performance.