Bag of Fisher Vectors representation of images by saliency-based spatial partitioning

Abstract

In content-based image retrieval systems, visual content of the image is the criterion for measuring image similarity. We propose a method to solve the problem of loss of spatial information of objects when local descriptors from an image with multiple objects are aggregated to form a global representation. In our approach, after saliency-based spatial partitioning, local feature descriptors from distinct sub-regions are aggregated to form a bag of Fisher Vectors representation. This helps in suppressing the information from background clutter in scenes while forming the global descriptor. The retrieval performance was evaluated in synthetic and real datasets. The evaluation results show that the bag of Fisher Vectors representation gives better performance compared to baseline approach using Fisher Vectors.