EDSOD: An Encoder-Decoder, Diffusion-model, and Swin-Transformer-based Small Object Detector

Abstract

Small object detection (SOD) given aerial images suffers from an information imbalance across different feature scales. This makes it extremely challenging to perform accurate SOD. Existing methods, e.g., Feature Pyramid Network (FPN)-based algorithms, focus on extracting high-resolution and low-resolution semantic features from different convolution layers. However, in deeper convolution layers, semantic feature misalignment and the loss of key information are inevitable. To tackle such issues, this work proposes a new encoder-decoder-based SOD framework with a Diffusion Model and Swin Transformer given aerial images. First, we reformulate an SOD task as a Noise-to-Box process. We then construct an encoder-decoder-based framework by using a diffusion model and Swin Transformer for dynamic bounding box generation. We introduce a decoupling training and inferencing strategy to recognize and locate small objects accurately. We finally evaluate the proposed framework on several public benchmarks. The experimental results well show its better SOD performance than the state of the art. Code is available at https://github.com/BrainPotter/EDSOD.