Bridging Task Boundaries: Remote Sensing Image-Text Retrieval via Dictionary-Driven Adaptation

Abstract

Given image (or text), remote sensing image-text retrieval (RSITR) aims to retrieve corresponding text (or image) within diverse remote sensing data. However, due to the complex scenes and compact distribution of targets in remote sensing data, existing methods, particularly those leveraging large models like CLIP, often generate features with high intra-modal similarity and insufficient distinctive characteristics, thus resulting in suboptimal retrieval performance. To address these issues, we pro- pose a novel dictionary-based RSITR method that jointly models image and text feature estimation. Specifically, by incorporating a general dictionary and the corresponding sparse coefficients, our method more effectively captures the correlations between the learned features. Furthermore, we introduce adaptive weighted metric learning on a sample-by-sample basis to encourage the model to focus on more challenging negative samples, promoting fine-grained feature alignment. The extensive experiments on the RSICD and RSITMD datasets demonstrate the effectiveness of our method, demonstrating significant improvements in retrieval performance.