Dense Isometric Non-Rigid Shape-From-Motion Based on Graph Optimization and Edge Selection

Abstract

In this letter, we propose a novel framework for dense isometric non-rigid shape-from-motion (Iso-NRSfM) based on graph topology and edge selection. A weighted undirected graph, of which nodes, edges, and weighted values are respectively the images, the image warps, and the number of the common features, is built. An edge selection algorithm based on maximum spanning tree and sub-modular optimization is presented to pick out the well-connected sub-graph for the warps with multiple images. Using the infinitesimal planarity assumption, the Iso-NRSfM problem is formulated as a graph optimization problem with the virtual measurements, which are based on metric tensor and Christoffel Symbol, and the variables related to the derivatives of the constructed points along the surface. The solution of this graph optimization problem directly leads to the normal field of the shape. Then, using a separable iterative optimization method, we obtain the dense point cloud with texture corresponding to the deformable shape robustly. In the experiments, the proposed method outperforms existing work in terms of constructed accuracy, especially when there exists missing/appearing (changing) data, noisy data, and outliers.