Relative Viewpoint Estimation Based on Structured 3d Representation Alignment

Abstract

Relative viewpoint estimation is a fundamental problem in various image processing applications. Traditional estimation approaches can fail if sufficient appearance overlap is not observed between two images. Recent advances in 3D representation learning from images have made it possible to exploit the underlying 3D structure. In this paper, we propose a relative viewpoint estimation method using an end-to-end trainable network that learns structured 3D representations. In the proposed method, an independent coordinate system is set for each image in order to construct a structured 3D representation. This makes it possible to estimate the relative viewpoint by aligning those representations through coordinate transformations. Experimental results on the ShapeNet, Pix3D, and Thingi10K datasets demonstrated that the proposed method achieves accurate estimation even if there is not sufficient observable appearance overlap between the images.