Closing the Communication Loop for Robotic Failures: Multi-Turn, Behavior-Tree-Grounded Explanations with Large Language Models

Abstract

Robot failures during collaborative tasks can frustrate users and reduce trust. To address this, we developed a failure communication module that combines large language models (LLMs) with Behavior Trees (BTs) to generate interactive, context-aware explanations for task failures. The module supports three key processes: (1) initial (high/medium/low) leveled explanations, (2) interactive clarifications for user follow-up questions, and (3) explicit verification of user actions to close the recovery loop. By leveraging the BT structure and persistent interaction history, it generates responsive, multi-turn explanations and reduces redundancy for repeated failures. We implemented and evaluated this module in real-time robotic pick-and-place tasks as a user study with 33 participants across three high/medium/low explanation conditions. The user study showed that the module improved resolution rates for challenging failures and reduced resolution times for simpler failures, demonstrating the effectiveness of LLM-powered, BT-grounded explanations in human-robot collaboration (HRC).