Static-Scene Constrained Optimization for Matrix/Tensor-Decomposition-free Foreground-Background Separation

Abstract

We propose an efficient foreground-background separation (FBS) method for (possibly noisy) video data. Most existing FBS methods model the background as a low-rank component. However, this approach is computationally expensive because it requires matrix/tensor decomposition of high-dimensional videos. In this paper, we first introduce a new background model, named static scene constraint (SSC), to FBS. SSC plays a role in accurately capturing the static background by keeping the temporal gradient of the background component to zero. In addition, SSC is formulated as a convex constraint using differences in the temporal direction, which eliminates the need for matrix/tensor decomposition in optimization and significantly reduces the computational cost compared to existing low-rank-based background models. Second, we formulate the FBS problem as a convex optimization problem involving SSC and develop an efficient solver based on a preconditioned primal-dual splitting algorithm, which can automatically determine the appropriate stepsizes based on problem structure. Finally, we demonstrate the efficiency and effectiveness of our method compared with state-of-the-art FBS methods through experiments using infrared and electron microscope videos.