Abstract:When a user takes a photo, a small motion of image frames is usually induced by hand shaking. On the one hand, the small motion contains parallax information, which is valuable for scene depth perception and can be potentially used in many applications, such as VR/AR and photo refocusing. On the other hand, due to narrow baselines, corresponding point matching of images is sensitive to noise, as a result of which scene reconstruction from uncalibrated small motion clips is quite challenging. Existing state-of-the-art methods for 3D reconstruction from small motion clips are generally less accurate since they do not consider the uncertainties. In this study, we propose a high-accuracy method for 3D reconstruction from uncalibrated small motion clips. The proposed method consists of two key steps. Firstly, in the self-calibration stage, we propose a viewpoint-weighted bundle adjustment method that fully considers the matching uncertainties of different neighboring viewpoints due to different baselines and assigns smaller confidence to the viewpoints with narrower baselines, thereby keeping the robustness during self-calibration. Furthermore, we present a TGV-based depth image estimation method that can alleviate noise caused by narrow baselines while maintaining slanted structures and detailed geometric features. The quantitative and qualitative experiments on public datasets and synthetic datasets clearly demonstrate the effectiveness of the proposed method in comparison with state-of-the-arts.