Combining Runtime Monitoring and Machine Learning with Human Feedback
Abstract
State-of-the-art machine-learned controllers for autonomous systems demonstrate unbeatable performance in scenarios known from training. However, in evolving environments---changing weather or unexpected anomalies---, safety and interpretability remain the greatest challenges for autonomous systems to be reliable and are the urgent scientific challenges. Existing machine-learning approaches focus on recovering lost performance but leave the system open to potential safety violations. Formal methods address this problem by rigorously analysing a smaller representation of the system but they rarely prioritize performance of the controller. We propose to combine insights from formal verification and runtime monitoring with interpretable machine-learning design for guaranteeing reliability of autonomous systems.
BibTeX
@article{Lukina_2024, title={Combining Runtime Monitoring and Machine Learning with Human Feedback}, volume={37}, url={https://ojs.aaai.org/index.php/AAAI/article/view/26815}, DOI={10.1609/aaai.v37i13.26815}, abstractNote={State-of-the-art machine-learned controllers for autonomous systems demonstrate unbeatable performance in scenarios known from training. However, in evolving environments---changing weather or unexpected anomalies---, safety and interpretability remain the greatest challenges for autonomous systems to be reliable and are the urgent scientific challenges. Existing machine-learning approaches focus on recovering lost performance but leave the system open to potential safety violations. Formal methods address this problem by rigorously analysing a smaller representation of the system but they rarely prioritize performance of the controller. We propose to combine insights from formal verification and runtime monitoring with interpretable machine-learning design for guaranteeing reliability of autonomous systems.}, number={13}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Lukina, Anna}, year={2024}, month={Jul.}, pages={15448-15448} }