Joint Feature and Kernel Fusion for Improved Depth-Aware Panoptic Segmentation

Abstract

Depth-aware Panoptic Segmentation, which combines panoptic segmentation and monocular depth estimation, is a challenging task that requires a comprehensive understanding of both scene geometry and object semantics. Recent multi-task learning approaches have leveraged dynamic kernel methods to tackle these tasks simultaneously. However, these methods often treat feature extraction and kernel generation for each task in isolation, failing to fully exploit the rich interdependencies between depth and semantic information. To address this, we propose a novel framework with Cross-Task Feature Fusion and Kernel Fusion mechanisms to enhance Depth-aware Panoptic Segmentation. Our approach enables deeper integration of features and kernels, promoting more effective information exchange and mutual reinforcement between tasks. Experiments show that our method brings significant improvements, demonstrating the potential of a more integrated multi-task learning strategy for Depth-aware Panoptic Segmentation.