基于风格迁移的双向孪生网络遥感变化检测
作者:
基金项目:

国家重点研发计划 (2018YFC0823002); “十四五”重点研发计划 (2021YFB3901105); 中央高校基本科研业务费专项 (FRF-TP-20-10B); 2022年度河南省重大科技专项 (221100210600); 重点实验室项目 (2022-JCJQ-LA-001-080)


Bi-directional Siamese Network for Remote Sensing Change Detection Based on Style Transfer
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [36]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    不同时间获取的同一区域的双时相遥感图像在风格上往往具有很大差异, 大多数研究方法忽略了这个问题, 导致在风格多样的数据集上应用时, 模型的性能指标和可视化效果不尽如人意. 为此, 本文首先使用风格迁移模块针对某一时刻原始图像生成类似另一时刻风格的风格迁移图像. 其次, 提出了一种基于双向风格迁移的孪生对称差分特征金字塔网络 (symmetrical difference feature pyramid network, SDFPNet), 确定不同风格迁移方向对变化检测精度提升的影响程度. 具体来说, 将原始图像和风格迁移图像作为SDFPNet输入, 使用两个孪生的轻量化网络和差分特征金字塔网络 (difference feature pyramid network, DFPNet)同时进行参数优化, 得到两个并行分支预测的变化图. 为了减少变化像素点的误判, 融合两个预测结果提升变化检测的准确性. 在LEVIR-CD、CDD和SYSU-CD这3个数据集上通过实验证明, 本文提出的基于双向风格迁移的SDFPNet在遥感变化检测任务上的评价指标优于SOTA (state-of-the-art)方法. 尤其是在由于季节变化, 风格差异较大的CDD和SYSU-CD数据集, 在CDD数据集上检测精度达到99.37%, F2分数达到94.19%, SYSU-CD数据集上检测精度达到92.31%. 有效解决了双时相图像风格差异大导致的变化检测精度不佳问题.

    Abstract:

    A majority of research methods neglect significant variations exhibited in the style of bi-temporal remote sensing images acquired at different times for the same area, leading to unsatisfactory model performance indexes and visualization when the model is applied to stylistically diverse datasets. To address this issue, a style transfer module is used in this article to generate an image with a style similar to that of another moment for the original image at a certain moment. Subsequently, a symmetrical difference feature pyramid network (SDFPNet) based on bi-directional style transfer is proposed to determine the influence degree of different style transfer directions on the improvement of change detection accuracy. Specifically, two lightweight Siamese networks and difference feature pyramid network (DFPNet) are used to conduct parameter optimization on the inputted original and stylized as SDFPNet, producing the change maps predicted by two parallel branches. To reduce the misclassification of changed pixels, the two prediction results are merged to improve the accuracy of change detection. Experiments on three datasets, LEVIR-CD, CDD, and SYSU-CD, demonstrate that the proposed SDFPNet based on bi-directional style transfer outperforms state-of-the-art (SOTA) methods in remote sensing change detection task, with results of CDD and SYSU-CD datasets more convincing, which have large style differences due to seasonal changes. The detection accuracy reaches 99.37% and F2 score reaches 94.19% on the CDD dataset, and the detection accuracy reaches 92.31% on the SYSU-CD dataset. The proposed method in this article effectively solves the problem of poor change detection accuracy caused by large style differences in bi-temporal images.

    参考文献
    [1] Singh A. Review article digital change detection techniques using remotely-sensed data. International Journal of Remote Sensing, 1989, 10(6): 989–1003.
    [2] Liang BQ, Weng QH. Assessing urban environmental quality change of Indianapolis, United States, by the remote sensing and GIS integration. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2011, 4(1): 43–55.
    [3] Li HX, Xiao PF, Feng XZ, et al. Using land long-term data records to map land cover changes in China over 1981–2010. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(4): 1372–1389.
    [4] Zelinski ME, Henderson J, Smith M. Use of Landsat 5 for change detection at 1998 Indian and Pakistani nuclear test sites. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(8): 3453–3460.
    [5] Hu JR, Zhang YZ. Seasonal change of land-use/land-cover (LULC) detection using MODIS data in rapid urbanization regions: A case study of the Pearl River delta region (China). IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2013, 6(4): 1913–1920.
    [6] Vogelmann JE, Kost JR, Tolk B, et al. Monitoring landscape change for LANDFIRE using multi-temporal satellite imagery and ancillary data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2011, 4(2): 252–264.
    [7] Li ZB, Shi WZ, Myint SW, et al. Semi-automated landslide inventory mapping from bitemporal aerial photographs using change detection and level set method. Remote Sensing of Environment, 2016, 175: 215–230.
    [8] Chen J, Yuan ZY, Peng J, et al. DASNet: Dual attentive fully convolutional Siamese networks for change detection in high-resolution satellite images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 1194–1206.
    [9] Tang X, Zhang TX, Ma JJ, et al. WNet: W-shaped hierarchical network for remote-sensing image change detection. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5615814.
    [10] Khelifi L, Mignotte M. Deep learning for change detection in remote sensing images: Comprehensive review and meta-analysis. IEEE Access, 2020, 8: 126385–126400.
    [11] Zhan Y, Fu K, Yan ML, et al. Change detection based on deep Siamese convolutional network for optical aerial images. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10): 1845–1849.
    [12] Peng DF, Zhang YJ, Guan HY. End-to-end change detection for high resolution satellite images using improved UNet++. Remote Sensing, 2019, 11(11): 1382.
    [13] Jiang HW, Hu XY, Li K, et al. PGA-SiamNet: Pyramid feature-based attention-guided Siamese network for remote sensing orthoimagery building change detection. Remote Sensing, 2020, 12(3): 484.
    [14] Huang JR, Shen Q, Wang M, et al. Multiple attention Siamese network for high-resolution image change detection. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5406216.
    [15] Sun ST, Mu L, Wang LZ, et al. L-UNet: An LSTM network for remote sensing image change detection. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8004505.
    [16] Chen HRX, Wu C, Du B, et al. Change detection in multisource VHR images via deep Siamese convolutional multiple-layers recurrent neural network. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(4): 2848–2864.
    [17] Chen H, Qi ZP, Shi ZW. Remote sensing image change detection with Transformers. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5607514.
    [18] Zhang C, Wang LJ, Cheng SL, et al. SwinSUNet: Pure Transformer network for remote sensing image change detection. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5224713.
    [19] Liu Z, Lin YT, Cao Y, et al. Swin Transformer: Hierarchical vision Transformer using shifted windows. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021. 9992–10002.
    [20] Chen H, Li WY, Shi ZW. Adversarial instance augmentation for building change detection in remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5603216.
    [21] Zhao WZ, Chen X, Ge XS, et al. Using adversarial network for multiple change detection in bitemporal remote sensing imagery. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8003605.
    [22] Portilla J, Simoncelli EP. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision, 2000, 40(1): 49–70.
    [23] Kang JJ, Zhu ZH. A unified energy-based control framework for tethered spacecraft deployment. Nonlinear Dynamics, 2019, 95(3): 1117–1131.
    [24] Reinhard E, Adhikhmin M, Gooch B, et al. Color transfer between images. IEEE Computer Graphics and Applications, 2001, 21(5): 34–41.
    [25] Li C, Wand M. Combining Markov random fields and convolutional neural networks for image synthesis. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016. 2479–2486.
    [26] Li C, Wand M. Precomputed real-time texture synthesis with Markovian generative adversarial networks. Proceedings of the 14th European Conference on Computer Vision. Amsterdam: Springer, 2016. 702–716.
    [27] Huang X, Belongie S. Arbitrary style transfer in real-time with adaptive instance normalization. Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017. 1510–1519.
    [28] Zhu JY, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017. 2242–2251.
    [29] Isola P, Zhu JY, Zhou TH, et al. Image-to-image translation with conditional adversarial networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017. 5967–5976.
    [30] Park T, Efros AA, Zhang R, et al. Contrastive learning for unpaired image-to-image translation. Proceedings of the 16th European Conference on Computer Vision. Glasgow: Springer, 2020. 319–345.
    [31] Lebedev MA, Vizilter YV, Vygolov OV, et al. Change detection in remote sensing images using conditional adversarial networks. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018, 42: 565–571.
    [32] Shi Q, Liu MX, Li SC, et al. A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5604816.
    [33] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. Proceedings of the 18th International Conference on Medical Image Computing and Computer-assisted Intervention. Munich, Germany: Springer, 2015. 234–241.
    [34] Zhou ZW, Siddiquee MR, Tajbakhsh N, et al. UNet++: A nested U-Net architecture for medical image segmentation. Proceedings of the 4th International Workshop on Deep Learning in Medical Image Analysis. Granada: Springer, 2018. 3–11.
    [35] Chen LC, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587, 2017.
    [36] Fang S, Li KY, Shao JY, et al. SNUNet-CD: A densely connected Siamese network for change detection of VHR images. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8007805.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

史经业,罗雅露,张梦鸽,支瑞聪,刘吉强.基于风格迁移的双向孪生网络遥感变化检测.计算机系统应用,2025,34(4):76-89

复制
分享
文章指标
  • 点击次数:88
  • 下载次数: 274
  • HTML阅读次数: 27
  • 引用次数: 0
历史
  • 收稿日期:2024-10-10
  • 最后修改日期:2024-10-30
  • 在线发布日期: 2025-02-25
文章二维码
您是第11482071位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号