Neural-Link: Non-Overlapping MPC Fusion and Passive Inertial Sensing on Soft Platforms

Abstract

Soft, elastic platforms may pose an intricate challenge towards sensor fusion as forces acting on the structure render extrinsic transformations variable over time. The present paper tackles this problem by introducing an elastic deformation model and embedding it into a sensor fusion scheme. The core of our method is given by a neural representation mapping temporal deformation sequences onto mass-normalized restoring forces. By using continuous time trajectory models as well as Newton’s second law, the sensor fusion problem becomes solvable by enforcing the consistency between second-order trajectory differentials and network outputs. The approach is validated on a loosely-coupled, real-world fusion scenario: an elastically connected, non-overlapping stereo camera system. As demonstrated, our approach permits relative camera alignment, absolute scale recovery, as well as inertial alignment from individual visual odometry results.<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>