Regional deep feature aggregation for image retrieval

Abstract

This paper presents a method to aggregate deep features for an object-based image retrieval system. Several recent works have demonstrated that it is quite important to selectively aggregate features with a weighting scheme and extract features from the limited regions likely to contain specific objects. Hence, the proposed method is to find possible candidate regions in an image, extract region descriptors from each region, and match images in a region-by-region manner. To adhere to using a pre-trained network without retraining or spatial verification, several candidate regions are found in an image and a more sophisticated pooling scheme is used for better performance. Specifically, salient points with active responses are detected in the image and clustered to form the candidate regions. In each region, we aggregate activations of a convolutional layer with the emphasis on more active spatial positions, and generate region descriptors effective for the object-based image retrieval. Our experiments show that the proposed method performs well on several public datasets, especially for the images showing the varied shapes or positions of an object.