A Reinforcement Learning Agent Controlled Multi-branch Small Object Detection Framework

Abstract

The past few years have witnessed the immense development of small object detection, which is aimed at detecting size-limited targets in high-resolution images. The prevailing methods focus on extracting fine-grained information by expanding the receptive fields and then generating the potential small object region. However, these solutions inevitably add sophisticated detectors and extra learning components, which incur time-consuming and computation-costing. Meanwhile, we observe that it’s suboptimal to extract fine features in such a generic way. To alleviate the issues, we propose a multi-branch small object detection framework with a regular-scale detection branch and a small-scale detection branch. Specifically, we design and pre-train a reinforcement learning agent to control feature extractors in both branches according to the results of small object areas. Moreover, we present a region clipping algorithm to rebuild the small object to regular size, which can be input into a mature detector directly. The extensive experiments on COCO, VisDrone, SODA-D, and our collecting TVDS datasets demonstrate our method outperforms the state-of-the-art methods in several metrics.