DFE3D: 双重特征增强的三维点云类增量学习
作者:
基金项目:

科技创新2030—“新一代人工智能”重大项目(2021ZD0112200)


DFE3D: Class-incremental Learning for 3D Point Cloud with Dual Feature Enhancement
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [44]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    随着点云采集技术的发展和三维应用需求的增加, 实际场景要求针对流动数据持续动态地更新点云分析网络. 对此, 提出了双重特征增强的三维点云类增量学习方法, 通过增量学习使点云目标分类技术能够适应新数据中不断出现新类别目标的场景. 该方法通过对点云数据特性和旧类信息的研究分别提出了差异性局部增强模块和知识注入网络, 以缓解类增量学习中的新类偏好问题. 具体而言, 差异性局部增强模块通过感知丰富的局部语义, 表征出三维点云物体中不同的局部结构特性. 随后, 根据目标中每个局部结构的全局信息获得各个局部的重要性权重, 强化对差异性局部特征的感知, 从而提高新旧类特征差异性. 另外, 知识注入网络将旧模型中的旧知识注入新模型的特征学习过程中, 增强后的混合特征能够更有效缓解旧类信息不足导致的新类偏好加剧现象. 在三维点云数据集ModelNet40, ScanObjectNN, ScanNet, ShapeNet上的实验表明, 该方法与现有最优方法相比, 在4个数据集上的平均增量准确率有2.03%、2.18%、1.65%、1.28% 提升.

    Abstract:

    As point cloud acquisition technology develops and the demand for 3D applications increases, real-world scenarios require continuous and dynamic updating of the point cloud analysis network with streaming data. This study proposes a dual feature enhancement for the class-incremental 3D point cloud object learning method, which adapts point cloud object classification to scenarios where new category objects keep emerging in newly acquired data through incremental learning. This study proposes a discriminative local enhancement module and knowledge injection network respectively to alleviate new class bias problems in class-incremental learning by studying the characteristics of point cloud data and old class information. Specifically, the discriminative local enhancement module characterizes the various local structural characteristics of 3D point cloud objects by perceiving expressive local features. Subsequently, the importance weights of each local structure are obtained based on the global information of each local structure, enhancing the perception of differential local features and improving the differentiation of new and old class features. Furthermore, the knowledge injection network injects old knowledge from the old model into the feature learning process of the new model. The enhanced hybrid features can effectively mitigate the increased new class bias caused by the lack of old class information. Under the incremental learning experimental settings of the 3D point cloud datasets ModelNet40, ScanObjectNN, ScanNet, and ShapeNet, extensive experiments show that compared with existing state-of-art methods, the method in this study has an average incremental accuracy improvement of 2.03%, 2.18%, 1.65%, and 1.28% on the four datasets.

    参考文献
    [1] Hamdi A, Giancola S, Ghanem B. MVTN: Multi-view transformation network for 3D shape recognition. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021. 1–11.
    [2] Riegler G, Osman Ulusoy A, Geiger A. OctNet: Learning deep 3D representations at high resolutions. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017. 6620–6629.
    [3] Qi CR, Su H, Mo KC, et al. PointNet: Deep learning on point sets for 3D classification and segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017. 77–85.
    [4] Qi CR, Yi L, Su H, et al. PointNet++: Deep hierarchical feature learning on point sets in a metric space. Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 5105–5114.
    [5] 许翔, 帅惠, 刘青山. 基于卦限卷积神经网络的3D点云分析. 自动化学报, 2021, 47(12): 2791–2800.
    [6] McCloskey M, Cohen NJ. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 1989, 24: 109–165.
    [7] 周大蔚, 汪福运, 叶翰嘉, 等. 基于深度学习的类别增量学习算法综述. 计算机学报, 2023, 46(8): 1577–1605.
    [8] Rebuffi SA, Kolesnikov A, Sperl G, et al. iCaRL: Incremental classifier and representation learning. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017. 5533–5542.
    [9] Kirkpatrick J, Pascanu R, Rabinowitz N, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences of the United States of America, 2017, 114(13): 3521–3526.
    [10] Douillard A, Cord M, Ollion C, et al. PODNet: Pooled outputs distillation for small-tasks incremental learning. Proceedings of the 16th European Conference on Computer Vision. Glasgow: Springer, 2020. 86–102.
    [11] 张子颖, 王敏. 基于Faster R-CNN和增量学习的车辆目标检测. 计算机系统应用, 2020, 29(2): 181–186.
    [12] Liu YY, Cong Y, Sun G, et al. L3DOC: Lifelong 3D object classification. IEEE Transactions on Image Processing, 2021, 30: 7486–7498.
    [13] Grossberg S. Consciousness CLEARS the mind. Neural Networks, 2007, 20(9): 1040–1053.
    [14] Dong JH, Cong Y, Sun G, et al. I3DOL: Incremental 3D object learning without catastrophic forgetting. Proceedings of the 35th AAAI Conference on Artificial Intelligence. AAAI, 2021. 6066–6074.
    [15] Zamorski M, Stypułkowski M, Karanowski K, et al. Continual learning on 3D point clouds with random compressed rehearsal. Computer Vision and Image Understanding, 2023, 228: 103621.
    [16] Wei X, Yu RX, Sun J. Learning view-based graph convolutional network for multi-view 3D shape analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(6): 7525–7541.
    [17] Wang PS, Liu Y, Guo YX, et al. O-CNN: Octree-based convolutional neural networks for 3D shape analysis. ACM Transactions on Graphics, 2017, 36(4): 72.
    [18] Graham B, Engelcke M, van der Maaten L. 3D Semantic segmentation with submanifold sparse convolutional networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018. 9224–9232.
    [19] Choy C, Gwak JY, Savarese S. 4D Spatio-temporal ConvNets: Minkowski convolutional neural networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 3070–3079.
    [20] 孟繁林, 何晓曦, 刘应浒, 等. 基于自注意力机制的点云分类分割. 计算机系统应用, 2024, 33(1): 177–184.
    [21] Wang Y, Sun YB, Liu ZW, et al. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics, 2019, 38(5): 146.
    [22] Xu MT, Ding RY, Zhao HS, et al. PAConv: Position adaptive convolution with dynamic kernel assembling on point clouds. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021. 3172–3181.
    [23] Zhao HS, Jiang L, Jia JY, et al. Point Transformer. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021. 16239–16248.
    [24] Liu YH, Tian B, Lv YS, et al. Point cloud classification using content-based Transformer via clustering in feature space. IEEE/CAA Journal of Automatica Sinica, 2024, 11(1): 231–239.
    [25] Dhar P, Singh RV, Peng KC, et al. Learning without memorizing. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 5133–5141.
    [26] Gao QK, Zhao C, Ghanem B, et al. R-DFCIL: Relation-guided representation learning for data-free class incremental learning. Proceedings of the 17th European Conference on Computer Vision. Tel Aviv: Springer, 2022. 423–439.
    [27] Smith JS, Karlinsky L, Gutta V, et al. CODA-Prompt: COntinual decomposed attention-based prompting for rehearsal-free continual learning. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023. 11909–11919.
    [28] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. arXiv:1503.02531, 2015.
    [29] Wu Y, Chen YP, Wang LJ, et al. Large scale incremental learning. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 374–382.
    [30] Zhao BW, Xiao X, Gan GJ, et al. Maintaining discrimination and fairness in class incremental learning. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 13205–13214.
    [31] Hou SH, Pan XY, Loy CC, et al. Learning a unified classifier incrementally via rebalancing. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 831–839.
    [32] Ahn H, Kwak J, Lim S, et al. SS-IL: Separated Softmax for incremental learning. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021. 824–833.
    [33] Qiu BL, Li HL, Wen HT, et al. CafeBoost: Causal feature boost to eliminate task-induced bias for class incremental learning. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023. 16016–16025.
    [34] Prabhu A, Torr PHS, Dokania PK. GDumb: A simple approach that questions our progress in continual learning. Proceedings of the 16th European Conference on Computer Vision. Glasgow: Springer, 2020. 524–540.
    [35] Aljundi R, Lin M, Goujaud B, et al. Gradient based sample selection for online continual learning. Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019. 1058.
    [36] Tiwari R, Killamsetty K, Iyer R, et al. GCR: Gradient coreset based replay buffer selection for continual learning. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022. 99–108.
    [37] Simon C, Koniusz P, Harandi M. On learning the geodesic path for incremental learning. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021. 1591–1600.
    [38] Ashok A, Joseph KJ, Balasubramanian VN. Class-incremental learning with cross-space clustering and controlled transfer. Proceedings of the 17th European Conference on Computer Vision. Tel Aviv: Springer, 2022. 105–122.
    [39] Hu ZY, Li YS, Lyu JC, et al. Dense network expansion for class incremental learning. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023. 11858–11867.
    [40] Gao XY, He YH, Dong SL, et al. DKT: Diverse knowledge transfer Transformer for class incremental learning. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023. 24236–24245.
    [41] Wu ZR, Song SR, Khosla A, et al. 3D ShapeNets: A deep representation for volumetric shapes. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015. 1912–1920.
    [42] Yi L, Kim VG, Ceylan D, et al. A scalable active framework for region annotation in 3D shape collections. ACM Transactions on Graphics, 2016, 35(6): 210.
    [43] Uy MA, Pham QH, Hua BS, et al. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019. 1588–1597.
    [44] Dai A, Chang AX, Savva M, et al. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017. 2432–2443.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

孙昊,帅惠,许翔,刘青山. DFE3D: 双重特征增强的三维点云类增量学习.计算机系统应用,2024,33(8):132-144

复制
分享
文章指标
  • 点击次数:302
  • 下载次数: 791
  • HTML阅读次数: 457
  • 引用次数: 0
历史
  • 收稿日期:2024-02-05
  • 最后修改日期:2024-03-05
  • 在线发布日期: 2024-06-28
文章二维码
您是第11249422位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号