Hyneter: Hybrid Network Transformer for Object Detection

Abstract

In this paper, we point out that the essential differences between CNN-based and Transformer-based detectors, which cause worse performance of small object in Transformer-based methods, are the gap between local information and global dependencies in feature extraction and propagation. To address these differences, we propose a new vision Transformer, called Hybrid Network Transformer (Hyneter). Different from the divide and conquer strategy in previous methods, Hyneters consist of Hybrid Network Backbone (HNB) and Dual Switching module (DS), which integrate local information and global dependencies, and transfer them simultaneously. Based on the balance strategy, HNB extends the range of local information by embedding convolution layers into Transformer blocks, and DS adjusts excessive reliance on global dependencies outside the patch. Ablation studies illustrate that Hyneters surpass the state-of-the-art results on multiple vision tasks.