TY - JOUR

T1 - Refractive Pose Refinement

T2 - Generalising the Geometric Relation between Camera and Refractive Interface

AU - Hu, Xiao

AU - Lauze, François

AU - Pedersen, Kim Steenstrup

N1 - Publisher Copyright: © 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

PY - 2023

Y1 - 2023

N2 - In this paper, we investigate absolute and relative pose estimation under refraction, which are essential problems for refractive structure from motion. To cope with refraction effects, we first formulate geometric constraints for establishing iterative algorithms to optimize absolute and relative pose. By classifying two scenarios according to the geometric relationship between the camera and refractive interface, we derive the corresponding solutions to solve the optimization problems efficiently. In the scenario where the geometry between the camera and refractive interface is fixed (e.g., underwater imaging), we also show that the refractive epipolar constraint for relative pose can be established as a summation of the classical essential matrix and two correction terms caused by refraction by using the virtual camera transformation. Thanks to its succinct form, the resulting refractive epipolar constraint can be efficiently optimized. We evaluate our proposed algorithms on synthetic data showing superior accuracy and computational efficiency compared to state-of-the-art (SOTA) methods. We further demonstrate the application of the proposed algorithms in refractive structure from motion on real data. Our datasets (Hu et al., RefractiveSfM, https://github.com/diku-dk/RefractiveSfM, 2022) and code (Hu et al., DIKU Refractive Scenes Dataset 2022, Data, 2022) are publicly available.

AB - In this paper, we investigate absolute and relative pose estimation under refraction, which are essential problems for refractive structure from motion. To cope with refraction effects, we first formulate geometric constraints for establishing iterative algorithms to optimize absolute and relative pose. By classifying two scenarios according to the geometric relationship between the camera and refractive interface, we derive the corresponding solutions to solve the optimization problems efficiently. In the scenario where the geometry between the camera and refractive interface is fixed (e.g., underwater imaging), we also show that the refractive epipolar constraint for relative pose can be established as a summation of the classical essential matrix and two correction terms caused by refraction by using the virtual camera transformation. Thanks to its succinct form, the resulting refractive epipolar constraint can be efficiently optimized. We evaluate our proposed algorithms on synthetic data showing superior accuracy and computational efficiency compared to state-of-the-art (SOTA) methods. We further demonstrate the application of the proposed algorithms in refractive structure from motion on real data. Our datasets (Hu et al., RefractiveSfM, https://github.com/diku-dk/RefractiveSfM, 2022) and code (Hu et al., DIKU Refractive Scenes Dataset 2022, Data, 2022) are publicly available.

KW - Pose estimation

KW - Reconstruction

KW - Refraction

KW - SfM

U2 - 10.1007/s11263-023-01763-4

DO - 10.1007/s11263-023-01763-4

M3 - Journal article

AN - SCOPUS:85149239061

VL - 131

SP - 1448

EP - 1476

JO - International Journal of Computer Vision

JF - International Journal of Computer Vision

SN - 0920-5691

IS - 6

ER -