Pedro Pinto Santos — accepted AI-conference papers

2026

Solving General-Utility Markov Decision Processes in the Single-Trial Regime with Online Planning

Pedro Pinto Santos, Alberto Sardinha, Francisco S. Melo

ICLR 2026poster

In this work, we contribute the first approach to solve infinite-horizon discounted general-utility Markov decision processes (GUMDPs) in the single-trial regime, i.e., when the agent's performance is evaluated based on a single trajectory. First, we provide some fundamental results regarding policy…

Cited by 0 Source ↗Scholar ↗

2025

The Number of Trials Matters in Infinite-Horizon General-Utility Markov Decision Processes

Pedro Pinto Santos, Alberto Sardinha, Francisco S. Melo

ICML 2025spotlight

The general-utility Markov decision processes (GUMDPs) framework generalizes the MDPs framework by considering objective functions that depend on the frequency of visitation of state-action pairs induced by a given policy. In this work, we contribute with the first analysis on the impact of the numb…

Cited by 0 Source ↗PDF ↗Scholar ↗