基于深度学习的单视图三维重建
作者:
基金项目:

国家自然科学基金(61370003)


Single-view 3D Reconstruction Based on Deep Learning
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [21]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    单视图三维重建在计算机视觉领域中是一个具有挑战性的问题. 为了提升现有三维重建算法重建后三维模型的精度, 本文除了提取图像全局特征之外还提取图像局部特征, 结合全局特征和局部特征并选取SDF (signed distance function)作为重建后的三维物体表达方式, 不仅提高了模型的精度, 生成了更高质量的3D形状, 还增强了模型的泛化能力, 使得深度模型可以以较高质量重建出其他物体种类. 实验结果表明, 本文提出的深度网络结构和3D形状表示方法与当今最先进的重建算法相比, 无论在重建后三维模型的效果还是新型物体的泛化中都有更好的表现.

    Abstract:

    Single-view 3D reconstruction is a challenging problem in computer vision. To improve the accuracy of the 3D model reconstructed by the existing 3D reconstruction algorithm, this study extracts both global and local features of the image. On this basis, the signed distance function (SDF) is used to describe the reconstructed 3D objects. In this way, high-quality 3D shapes are generated, and the model has higher accuracy and enhanced generalization capability, which enables the deep model to reconstruct other types of objects with high quality. Experiments demonstrate that compared with the most advanced reconstruction algorithm at present, the proposed deep network and the method for representing 3D shapes have better performance in the effects of reconstructed 3D models and the generalization of new objects.

    参考文献
    [1] Chang AX, Funkhouser T, Guibas L, et al. ShapeNet: An information-rich 3D model repository. arXiv: 1512.03012, 2015.
    [2] Wu JJ, Wang YF, Xue TF, et al. MarrNet: 3D shape reconstruction via 2.5D sketches. Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 540–550.
    [3] Fan HQ, Su H, Guibas L. A point set generation network for 3D object reconstruction from a single image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017. 2463–2471.
    [4] Mandikal P, Navaneet KL, Agarwal M, et al. 3D-LMNet: Latent embedding matching for accurate and diverse 3D point cloud reconstruction from a single image. Proceedings of British Machine Vision Conference 2018. Newcastle: BMVA Press, 2018. 1–19.
    [5] Zhang Y, Liu Z, Liu TP, et al. RealPoint3D: An efficient generation network for 3D object reconstruction from a single image. IEEE Access, 2019, 7: 57539–57549. [doi: 10.1109/ACCESS.2019.2914150
    [6] Choy CB, Xu DF, Gwak J, et al. 3D-R2N2: A unified approach for single and multi-view 3D object reconstruction. Proceedings of the 14th European Conference on Computer Vision. Amsterdam: Springer, 2016. 628–644.
    [7] Girdhar R, Fouhey DF, Rodriguez M, et al. Learning a predictable and generative vector representation for objects. Proceedings of the 14th European Conference on Computer Vision. Amsterdam: Springer, 2016. 484–499.
    [8] Dai A, Qi CR, Nießner M. Shape completion using 3D-encoder-predictor CNNs and shape synthesis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017. 6545–6554.
    [9] Pontes JK, Kong C, Sridharan S, et al. Image2Mesh: A learning framework for single image 3D reconstruction. Proceedings of 14th Asian Conference on Computer Vision. Perth: Springer, 2019. 365–381.
    [10] Groueix T, Fisher M, Kim VG, et al. A Papier-Mache approach to learning 3D surface generation. Proceedings of the IEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018. 216–224.
    [11] Jack D, Pontes JK, Sridharan S, et al. Learning free-form deformations for 3D object reconstruction. Proceedings of 14th Asian Conference on Computer Vision. Perth: Springer, 2019. 317–333.
    [12] Park JJ, Florence P, Straub J, et al. DeepSDF: Learning continuous signed distance functions for shape representation. Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 165–174.
    [13] Wang JR, Fang ZY. GSIR: Generalizable 3D shape interpretation and reconstruction. Proceedings of 16th European Conference on Computer Vision. Glasgow: Springer, 2020. 498–514.
    [14] Mescheder L, Oechsle M, Niemeyer M, et al. Occupancy networks: Learning 3D reconstruction in function space. Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019. 4455–4465.
    [15] Chen ZQ, Zhang H. Learning implicit fields for generative shape modeling. Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 5932–5941.
    [16] Wang WY, Xu QG, Ceylan D, et al. DISN: Deep implicit surface network for high-quality single-view 3D reconstruction. Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019. 45.
    [17] Thai A, Stojanov S, Upadhya V, et al. 3D reconstruction of novel object shapes from single image. Proceedings of 2021 International Conference on 3D Vision (3DV). London: IEEE, 2021. 85–95.
    [18] Kleineberg M, Fey M, Weichert F. Adversarial generation of continuous implicit shape representations. Proceedings of 41st Annual Conference of the European Association for Computer Graphics. Norrköping: Eurographics Association, 2020. 41–44.
    [19] Zhou Y, Barnes C, Lu JW, et al. On the continuity of rotation representations in neural networks. Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 5738–5746.
    [20] Lorensen WE, Cline HE. Marching cubes: A high resolution 3D surface construction algorithm. ACM SIGGRAPH Computer Graphics, 1987, 21(4): 163–169. [doi: 10.1145/37402.37422
    [21] Zhang XM, Zhang ZT, Zhang CK, et al. Learning to reconstruct shapes from unseen classes. Proceedings of the 32nd International Conference on Neural Information Processing Systems (NeurIPS). Montréal: Curran Associates Inc., 2018. 2263–2274.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

邹泞键,冯刚,陈卫东.基于深度学习的单视图三维重建.计算机系统应用,2022,31(9):300-305

复制
分享
文章指标
  • 点击次数:947
  • 下载次数: 2193
  • HTML阅读次数: 3631
  • 引用次数: 0
历史
  • 收稿日期:2021-12-10
  • 最后修改日期:2022-01-10
  • 在线发布日期: 2022-06-16
文章二维码
您是第11274133位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号