Interactive on-device Mobile Landmark Recognition with compact binary codes

Abstract

Interactive mobile vision applications, such as Mobile Landmark Recognition (MLR), have recently attracted ever increasing research attention due to the exponential growth of mobile devices. However, the recognition accuracy retains as a bottleneck hesitating the proliferation of such applications. To address this challenge, in this paper we design a novel framework based on interactive image segmentation and multiple visual features fusion to improve the accuracy of on-device MLR systems. Firstly, we propose a simple but effective vector binarization method to reduce the memory usage of image description significantly without decreasing the search accuracy. Secondly, we design a location aware fusion algorithm which can integrate multiple visual features into a compact yet discriminative image descriptor on-device. Thirdly, a user-friendly interaction scheme is developed to enable interactive foreground/background segmentation to improve the recognition accuracy. Experimental results demonstrate the effectiveness of the proposed algorithm for on-device MLR applications.