Enhancing DETR Efficiency with Inter-Object Relationship and Semantic Spectral Decomposition-Based Distillation

Abstract

DEtection TRansformer (DETR) is an innovative object detection framework based on transformer architecture that achieves end-to-end detection using a self-attention mechanism, moving away from traditional CNN-based approaches. However, the high computational cost of DETR models limits their deployment in practical applications. To address this issue, we propose a novel distillation framework specifically designed for DETR-based models, incorporating two key techniques: Inter-Object Relationship (IOR) distillation and Semantic Spectral Decomposition (SSD) distillation. The IOR distillation method captures and transfers complex dependencies between objects, enhancing the student model’s understanding of contextual relationships. In parallel, the SSD distillation method focuses on extracting and refining semantic information across spectral levels, improving the model’s focus on relevant features for more accurate detection. Extensive experiments on the MS COCO dataset demonstrate that our framework significantly enhances student model performance, achieving state-of-the-art results while effectively reducing the computational demands of DETR models without compromising accuracy.