AVP Scene Graph: Hierarchical Visual Language Mapping and Navigation for Autonomous Valet Parking
Xiangru Mu, Fengyi Chen, Runhan Wang, Siyuan Chen, Jiyuan Cai, Jia Cai, Ming Yang, Tong Qin
Abstract
Autonomous valet parking (AVP) aims to help the human drivers navigate to the desired location in the parking lot. Currently, the AVP task is not flexible enough to perform the open-vocabulary navigation tasks such as "navigate to the exit" or "park near the elevator". The widely used map formats for AVP like vectorized maps have some limitations including limited semantics, high cost and poor human-machine interaction, restricting the flexible application of AVP in complex scenarios. To address these problems, we propose AVP Scene Graph (AVP-SG), a hierarchical visual language mapping and navigation framework for open-vocabulary AVP tasks, which enables autonomous navigation from multi-modal human instructions. Our framework consists of two parts: a bottom-up mapping module and a top-down navigation module. In the mapping pipeline, assisted by the vision-language model (VLM) and optical character recognition (OCR) model, we first extract open-vocabulary conceptual semantics from images and project them to the elements of map. Next, by the bottom-up scheme performing feature fusion layer by layer, the scene graph is built hierarchically, consisting of slot, lane, block, and garage layer. In the top-down navigation pipeline, the navigation goal can be efficiently found by an LLM-enhanced graph retrieval approach. Experiments on real-world AVP tasks prove that the self-driving vehicle can perform open-vocabulary AVP tasks successfully utilizing the AVP-SG.
BibTeX
@inproceedings{iros2025_avpscenegraphhie,
title = {AVP Scene Graph: Hierarchical Visual Language Mapping and Navigation for Autonomous Valet Parking},
author = {Xiangru Mu and Fengyi Chen and Runhan Wang and Siyuan Chen and Jiyuan Cai and Jia Cai and Ming Yang and Tong Qin},
booktitle = {IROS 2025},
year = {2025}
}