UVSS: Unified Video Stabilization and Stitching for Surround View of Tractor-Trailer Vehicles

Abstract

Automotive surround-view camera systems have been commonly employed in automated driving to aid in near-field sensing and other perception tasks. Due to the large size of the body and the presence of multiple blind spots, panoramic surround-view systems are particularly crucial for tractor-trailer vehicles. However, the non-rigid body of tractor-trailer vehicles introduces pose changes between cameras, rendering traditional calibration-based methods inadequate. Additionally, cameras mounted separately on the tractor and the trailer will experience independent vibrations, resulting in undesirable shakiness in captured videos. In this paper, we propose a unified video stabilization and stitching method to address these challenges, which can smooth the unsteady frames and align the images from moving cameras. Delving into video stabilization techniques, we extend mesh-based motion model for unified stitching and leverage deep-learning based modules to handle complex real-world scenarios. Moreover, we design a new optimization framework to estimate the optimal displacements of mesh vertices, enabling simultaneous stabilization and stitching of frames. The experimental results, obtained by public datasets and videos captured from a model tractor-trailer vehicle, demonstrate that our approach outperforms previous methods and is highly effective in real-world applications.