Failure Detection With Zero-Shot Error Correction in Robotic Manipulation
Haojie Luo, Xizhou Bu, Hongbo Wang, Shijie Guo, Junxiu Liu, Wei Li
Abstract
Diffusion Policy (DP) is effective for imitation learning in robotic manipulation, yet likelihood-based replanning methods lack self-correction and often fail once execution deviates. Vision-Language Models (VLMs) offer strong spatial reasoning for failure handling but incur prohibitive latency for real-time use. We propose a two-stage Failure Detection and Error Correction (FDEC) framework: a lightweight visual dynamics model performs fast predictive monitoring to flag potential failures, and a threshold switch triggers a VLM for high-precision verification and adaptive correction only when needed. On RLBench and physical platforms, FDEC improves detection speed by 2.1× over pure VLM-based detectors and raises task success by 26.7%, demonstrating a practical trade-off between responsiveness and generalization for real-world manipulation.
BibTeX
@inproceedings{ral2026_failuredetection,
title = {Failure Detection With Zero-Shot Error Correction in Robotic Manipulation},
author = {Haojie Luo and Xizhou Bu and Hongbo Wang and Shijie Guo and Junxiu Liu and Wei Li},
booktitle = {RA-L 2026},
year = {2026}
}