Multi-scale Deformable 3D Medical Image Registration Based on Transformer
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [30]
  • |
  • Related
  • |
  • Cited by
  • | |
  • Comments
    Abstract:

    Deformable 3D medical image registration remains challenging due to irregular deformations of human organs. This study proposes a multi-scale deformable 3D medical image registration method based on Transformer. Firstly, the method adopts a multi-scale strategy to realize multi-level connections to capture different levels of information. Self-attention mechanism is employed to extract global features, and dilated convolution is used to capture broader context information and more detailed local features, so as to enhance the registration network’s fusion capacity for global and local features. Secondly, according to the sparse prior of the image gradient, the normalized total gradient is introduced as a loss function, effectively reducing the interference of noise and artifacts on the registration process, and better adapting to different modes of medical images. The performance of the proposed method is evaluated on publicly available brain MRI datasets (OASIS and LPBA). The results show that the proposed method can not only maintain the advantages of the learning-based method in run-time but also well performs in mean square error and structural similarity. In addition, ablation experiment results further prove the validity of the method and normalized total gradient loss function design proposed in this study.

    Reference
    [1] 郭艳芬, 崔喆, 杨智鹏, 等. 基于深度学习的医学图像配准技术研究进展. 计算机工程与应用, 2021, 57(15): 1–8.
    [2] 罗雅雯, 王远军. 基于特征提取的脑部医学图像配准研究. 生物医学工程学进展, 2023, 44(3): 226–234.
    [3] 邱玲娜, 吴经芝, 吴泳兴, 等. 不同CT-CBCT配准方法对颈胸膜固定方式下食管癌患者摆位误差的影响. 福建医科大学学报, 2023, 57(3): 217–222.
    [4] Ji HZ, Li YS, Dong EQ, et al. A non-rigid image registration method based on multi-level B-spline and L2-regularization. Signal, Image and Video Processing, 2018, 12(6): 1217–1225.
    [5] Chun SY, Fessler JA. A simple regularizer for B-spline nonrigid image registration that encourages local invertibility. IEEE Journal of Selected Topics in Signal Processing, 2009, 3(1): 159–169.
    [6] Fan JF, Cao XH, Yap PT, et al. BIRNet: Brain image registration using dual-supervised fully convolutional networks. Medical Image Analysis, 2019, 54: 193–206.
    [7] Sokooti H, de Vos B, Berendsen F, et al. Nonrigid image registration using multi-scale 3D convolutional neural networks. Proceedings of the 20th International Conference on Medical Image Computing and Computer-assisted Intervention. Quebec City: Springer, 2017. 232–239.
    [8] Balakrishnan G, Zhao A, Sabuncu MR, et al. VoxelMorph: A learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging, 2019, 38(8): 1788–1800.
    [9] Shi JC, He YT, Kong YY, et al. XMorpher: Full Transformer for deformable medical image registration via cross attention. Proceedings of the 25th International Conference on Medical Image Computing and Computer-assisted Intervention. Singapore: Springer, 2022. 217–226.
    [10] Kim B, Kim DH, Park SH, et al. CycleMorph: Cycle consistent unsupervised deformable image registration. Medical Image Analysis, 2021, 71: 102036.
    [11] Mok TCW, Chung ACS. Fast symmetric diffeomorphic image registration with convolutional neural networks. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 4643–4652.
    [12] 应时辉, 杨菀, 杜少毅, 等. 基于深度学习的医学影像配准综述. 模式识别与人工智能, 2021, 34(4): 287–299.
    [13] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing Systems. Long Beach: NIPS, 2017. 5998–6008.
    [14] Liu Z, Lin YT, Cao Y, et al. Swin Transformer: Hierarchical vision Transformer using shifted windows. Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021. 9992–10002.
    [15] Chen JY, Frey EC, He YF, et al. TransMorph: Transformer for unsupervised medical image registration. Medical Image Analysis, 2022, 82: 102615.
    [16] Avants BB, Epstein CL, Grossman M, et al. Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis, 2008, 12(1): 26–41.
    [17] Viola P, Wells III WM. Alignment by maximization of mutual information. International Journal of Computer Vision, 1997, 24(2): 137–154.
    [18] Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions. Proceedings of the 4th International Conference on Learning Representations. San Juan: ICLR, 2016.
    [19] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929, 2021.
    [20] 马明睿. 基于深度学习的医学影像配准算法研究 [博士学位论文]. 长春: 吉林大学, 2023.
    [21] Jaderberg M, Simonyan K, Zisserman A, et al. Spatial Transformer networks. Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal: MIT Press, 2015. 2017–2025.
    [22] Mok TCW, Chung ACS. Affine medical image registration with coarse-to-fine vision Transformer. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022. 20803–20812.
    [23] Chen SJ, Shen HL, Li CG, et al. Normalized total gradient: A new measure for multispectral image registration. IEEE Transactions on Image Processing, 2018, 27(3): 1297–1310.
    [24] Marcus DS, Wang TH, Parker J, et al. Open access series of imaging studies (OASIS): Cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. Journal of Cognitive Neuroscience, 2007, 19(9): 1498–1507.
    [25] Shattuck DW, Mirza M, Adisetiyo V, et al. Construction of a 3D probabilistic atlas of human cortical structures. NeuroImage, 2008, 39(3): 1064–1080.
    [26] Meng MY, Bi L, Fulham M, et al. Non-iterative coarse-to-fine Transformer networks for joint affine and deformable image registration. Proceedings of the 26th International Conference on Medical Image Computing and Computer-assisted Intervention. Vancouver: Springer, 2023. 750–760.
    [27] Dice LR. Measures of the amount of ecologic association between species. Ecology, 1945, 26(3): 297–302.
    [28] Taha AA, Hanbury A. An efficient algorithm for calculating the exact Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(11): 2153–2163.
    [29] Wang Z, Bovik AC, Sheikh HR, et al. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600–612.
    [30] Beg MF, Miller MI, Trouvé A, et al. Computing large deformation metric mappings via geodesic flows of diffeomorphisms. International Journal of Computer Vision, 2005, 61(2): 139–157.
    Related
    Cited by
Get Citation

陈璐莹,喻国荣,鲍海洲,边小勇,陈聪鹏.基于Transformer的多尺度可变形三维医学图像配准.计算机系统应用,2025,34(1):47-57

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:June 13,2024
  • Revised:July 10,2024
  • Online: November 15,2024
Article QR Code
You are the first992116Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063