Score-based Learning for Relevance Prediction in Image Similarity Search

Abstract

Predicting the performance of queries when labels are not present has been a recurring problem faced in information retrieval systems. Beyond its clear importance, it can also be applied to aid post-retrieval optimization approaches such as re-ranking or rank-aggregation. However, most post-retrieval performance prediction approaches to retrieval systems rely on generating a single effectiveness value of performance for queries. We propose an alternative method to assess the performance of systems reliant on similarity search, which consists of predicting the individual relevance of ranked results according to the distribution of similarity scores of a given query compared to instances in a collection. The idea is that relationships between the ith ranked score and other scores of the rank can be leveraged to generate features which, in turn, are used to classify ranked objects according to their relevance to the query. We propose a positional classification scheme, in conjunction with simple and fast score-based features to predict the relevance of the top-10 results of a similarity search rank. Our results in nine scenarios, comprising three different large image datasets, show good prediction accuracy for the top-10 results, with the advantage of being amenable suitable to deploy at query time.