Security-Aware Reinforcement Learning under Linear Temporal Logic Specifications
Bohan Cui, Keyi Zhu, Shaoyuan Li, Xiang Yin
Abstract
In this paper, we investigate the problem of reinforcement learning under linear temporal logic (LTL) specifications for Markov decision processes (MDPs) with security constraints. We consider an outside passive intruder (observer) that can observe the external output behavior of the system through an output projection. We assume that the secret of the system is a subset of the initial states. The security constraint requires that the observer can never infer for sure that the agent was initiated from a secret state. Our objective is to learn a control policy that achieves the LTL task while ensuring security. To solve the problem of shaping the reward for reinforcement learning, we propose an approach based on the initial-state estimator and the limit deterministic Büchi automata. We illustrate the proposed approach by a case study of mobile robot example.
BibTeX
@inproceedings{icra2023_securityawarerei,
title = {Security-Aware Reinforcement Learning under Linear Temporal Logic Specifications},
author = {Bohan Cui and Keyi Zhu and Shaoyuan Li and Xiang Yin},
booktitle = {ICRA 2023},
year = {2023}
}