EPI-Mamba: State Space Model for Semantic Segmentation from Light Fields

Abstract

Global contextual dependency is of significance for semantic segmentation from light fields. However, previous works mostly exploit attention mechanisms to model spatial context dependency and angular context dependency separately, since a light field capture is very data-intensive. Considering that self-attention is resource-consuming, existing methods only use it to capture spatial contextual dependency for the central view of light fields, and employ resource-efficient attention mechanisms to model angular view dependencies instead. Inspired by the recent success of State Space Sequence model (SSM), we present a novel SSM-based encoder for light field semantic segmentation, which models spatio-angular contextual dependency together. The proposed method is evaluated on the existing light field semantic segmentation datasets, and experimental results show that the proposed method significantly outperforms previous state-of-the-art methods. The code and models used in this work are available in https://github.com/YanWQ/EPI-Mamba