RelAIBotiX: Reliability Assessment for AI-Controlled Robotic Systems

Abstract

AI-controlled robotic systems can introduce significant risks to both humans and the environment. Traditional reliability assessment methods fall short in addressing the complexities of these systems, particularly when dealing with black-box or dynamically changing control policies. The traditional approaches are applied manually and do not consider frequent software updates. In this paper, we present RelAIBotiX, a new methodology that enables dynamic and continuous reliability assessment, specifically tailored for robotic systems controlled by AI algorithms. RelAIBotiX combines four methods: (i) Skill Detection that automatically identifies executed skills using deep learning techniques, (ii) Behavioral Analysis that creates an operational profile of the robotic system containing information about the skill execution sequence, active components for each skill, and their utilization intensity that influence their failure rate, (iii) Reliability Model Generation that automatically transforms the operational profile and reliability data of robotic hardware components into quantitative hybrid reliability models, and (iv) Reliability Model Solver for the numerical evaluation of the generated reliability models. Our evaluation included computing the reliability of the system, the probability of failure of individual skills, and component sensitivity analysis. We validated the applicability of the proposed framework across five simulative and real-world setups.