Do you see the bakery? Leveraging geo-referenced texts for global localization in public maps

Abstract

Text is one of the richest sources of information in an urban environment. Although textual information is heavily relied on by humans for a majority of the daily tasks, its usage has not been completely exploited in the field of robotics. In this work, we propose a localization approach utilizing textual features in urban environments. Starting at an unknown location, equipped with an RGB-camera and a compass, our approach uses off-the-shelf text extraction methods to identify text labels in the vicinity. We then apply a probabilistic localization approach with specific sensor models to integrate multiple observations. An extensive evaluation with real-world data gathered in different cities reveals an improvement over GPS-based localization when using our method.