基于分组ConvLSTM和Transformer的双分支遥感高光谱图像超分辨率网络
作者:
基金项目:

国家自然科学基金青年项目(32301702); 山东省自然科学基金青年项目(ZR2021QC120)


Dual-branch Remote Sensing Hyperspectral Image Super-resolution Network Based on Grouped ConvLSTM and Transformer
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [21]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    遥感高光谱图像超分辨率(remote sensing hyperspectral image single super-resolution, HSISR)任务近年来已取得可观进展, 其中使用深度卷积神经网络(convolutional neural network, CNN)技术的方法得到广泛运用. 然而, 大多数基于CNN的超分辨模型往往会忽略遥感高光谱图像的光谱结构, 同时由于卷积网络受卷积核大小限制, 长距离的特征依赖关系被忽略, 进而影响了重建的精度. 为了解决这些问题, 本文提出了一个基于分组ConvLSTM和Transformer的双分支遥感高光谱图像超分辨率网络(dual-branch remote sensing hyperspectral image super-resolution network based on grouped ConvLSTM and Transformer, DGCTNet), 该方法结合了Transformer捕捉长距离依赖关系和卷积长短时记忆网络(ConvLSTM)对提取序列性特征的优势, 在提取空间特征的同时保持了光谱的有序性, 增强了重建图像的效果. 此外, DGCTNet还设计了边缘学习网络, 将边缘信息扩散到图像空间中. 同时为重新校准光谱响应, 加入提出的双组级通道注意力机制(dual-group level channel self-attention, DSA). 在Houston数据集上的实验表明, DGCTNet方法在定量评价指标和多种场景下的视觉质量上, 都优于当前最先进的对比模型.

    Abstract:

    Remote sensing hyperspectral image single super-resolution (HSISR) tasks have made considerable progress in recent years. Methods using deep convolutional neural network (CNN) technology are widely employed. However, most CNN-based super-resolution models tend to ignore the spectral structure of remote sensing hyperspectral images. Meanwhile, due to the limitation of convolutional networks by the size of convolutional kernels, long-distance feature dependencies are ignored, which in turn affects the reconstruction accuracy. To solve these problems, this study proposes adual-branch remote sensing hyperspectral image super-resolution network based on grouped ConvLSTM and Transformer (DGCTNet), which combines the advantages of Transformer in capturing long-distance dependencies and ConvLSTM in extracting sequential features. It enhances the reconstructed image effect by extracting spatial features while maintaining spectral orderliness. In addition, DGCTNet also designs an edge learning network to diffuse edge information into the image space. At the same time, to recalibrate the spectral response, the proposed dual-group level channel self-attention mechanism (DSA) is added. Experiments on the Houston dataset show that the proposed DGCTNet method outperforms the current state-of-the-art comparison models in terms of quantitative evaluation metrics and visual quality in a wide variety of scenarios.

    参考文献
    [1] 谭克龙, 万余庆, 杨一德, 等. 高光谱遥感考古探索研究. 红外与毫米波学报, 2005, 24(6): 437–440.
    [2] 杨含成. 高光谱遥感技术在智慧农业中的应用研究. 南方农机, 2022, 53(20): 156–158.
    [3] 许洪, 王向军. 多光谱、超光谱成像技术在军事上的应用. 红外与激光工程, 2007, 36(1): 13–17.
    [4] Haris M, Shakhnarovich G, Ukita N. Recurrent back-projection network for video super-resolution. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 3892–3901.
    [5] 熊卫辉. 基于稀疏表示和字典学习的单帧图像超分辨率算法研究 [硕士学位论文]. 昆明: 昆明理工大学, 2013.
    [6] 路庆春. 正则化超分辨率图像重建算法研究 [硕士学位论文]. 合肥: 中国科学技术大学, 2010.
    [7] Yang JC, Wright J, Huang TS, et al. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 2010, 19(11): 2861–2873.
    [8] Dong C, Loy CC, He KM, et al. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307.
    [9] He KM, Zhang XY, Ren SQ, et al. Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016. 770–778.
    [10] Mei SH, Yuan X, Ji JY, et al. Hyperspectral image spatial super-resolution via 3D full convolutional neural network. Remote Sensing, 2017, 9(11): 1139.
    [11] Sun L, Li Y, Zheng M, et al. MCNet: Multiscale visible image and infrared image fusion network. Signal Processing, 2023, 208: 108996.
    [12] Li Y, Zhang L, Ding C, et al. Single hyperspectral image super-resolution with grouped deep recursive residual network. Proceedings of the 4th IEEE International Conference on Multimedia Big Data. Xi’an: IEEE, 2018. 1–4.
    [13] Jiang JJ, Sun H, Liu XM, et al. Learning spatial-spectral prior for super-resolution of hyperspectral imagery. IEEE Transactions on Computational Imaging, 2020, 6: 1082–1096.
    [14] 呼亚萍, 孔韦韦, 李萌, 等. 基于边缘检测全变分模型的图像去噪方法. 现代电子技术, 2021, 44(5): 52–56.
    [15] Hui YM, Wang J, Shi Y, et al. Low light image enhancement algorithm based on detail prediction and attention mechanism. Entropy, 2022, 24(6): 815.
    [16] Pan XR, Yang LN, Sun X, et al. Research on the extraction of hazard sources along high-speed railways from high-resolution remote sensing images based on TE-ResUNet. Sensors, 2022, 22(10): 3784.
    [17] Zhang TL, Chen HZ, Chen S, et al. Edge-enhanced efficient network for remote sensing image super-resolution. International Journal of Remote Sensing, 2022, 43(14): 5324–5347.
    [18] Xu YY, Luo W, Hu AN, et al. TE-SAGAN: An improved generative adversarial network for remote sensing super-resolution images. Remote Sensing, 2022, 14(10): 2425.
    [19] Jia JR, Ji LY, Zhao YC, et al. Hyperspectral image super-resolution with spectral-spatial network. International Journal of Remote Sensing, 2018, 39(22): 7806–7829.
    [20] Lu JJ, Tang CL. Short-time prediction of image motion trajectories based on convolutional long and short-term memory neural networks. Proceedings of the 5th IEEE International Conference on Civil Aviation Safety and Information Technology. Dali: IEEE, 2023. 1–6.
    [21] Jiang K, Wang ZY, Yi P, et al. Edge-enhanced GAN for remote sensing image superresolution. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(8): 5799–5812.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

边太成,杨锦,朱习军.基于分组ConvLSTM和Transformer的双分支遥感高光谱图像超分辨率网络.计算机系统应用,2025,34(3):286-295

复制
分享
文章指标
  • 点击次数:49
  • 下载次数: 768
  • HTML阅读次数: 3
  • 引用次数: 0
历史
  • 收稿日期:2024-08-20
  • 最后修改日期:2024-09-19
  • 在线发布日期: 2025-01-16
文章二维码
您是第10782479位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号