Geo-Supervised Visual Depth Prediction

Abstract

We propose using global orientation from inertial measurements, and the bias it induces on the shape of objects populating the scene, to inform visual three-dimensional reconstruction. We test the effect of using the resulting prior in-depth prediction from a single image, where the normal vectors to surfaces of objects of certain classes tend to align with gravity or be orthogonal to it. Adding such a prior to baseline methods for monocular depth prediction yields improvements beyond the state-of-the-art and illustrates the power of gravity as a supervisory signal.