Probabilistic Verification of Recurrent Neural Networks for Single and Multi-Agent Reinforcement Learning

Luca Marzari; Enrico Marchesini

IJCAI 20260 citations

Probabilistic Verification of Recurrent Neural Networks for Single and Multi-Agent Reinforcement Learning

Abstract

History-dependent policies induced by recurrent neural networks (RNNs) rely on latent hidden state dynamics, making verification in partially observable reinforcement learning (RL) challenging. Existing RNN verification tools typically rely on restrictive modeling assumptions or coarse over-approximations of the hidden state space, which can lead to overly conservative or inconclusive results. We propose RNN Probabilistic Verification (RNN-ProVe), a probabilistic framework that estimates the likelihood of undesired behaviors in RNN-based policies. RNN-ProVe uses policy-driven sampling to approximate the set of hidden states that are feasible under a trained policy, and derives statistical error bounds to produce bounded-error, high-confidence estimates of behavioral violations. Experiments on partially observable single-agent and cooperative multi-agent tasks show that RNN-ProVe yields more quantitative, feasibility-aware probabilistic guarantees than existing tools, while scaling to recurrent and multi-agent settings.

Agent-based and Multi-agent Systems: Formal verification, validation and synthesisAI Ethics, Trust, Fairnes: Safety and robustnessMachine Learning: Reinforcement learning

BibTeX

@inproceedings{ijcai2026_probabilisticver,
  title = {Probabilistic Verification of Recurrent Neural Networks for Single and Multi-Agent Reinforcement Learning},
  author = {Luca Marzari and Enrico Marchesini},
  booktitle = {IJCAI 2026},
  year = {2026}
}