An Analytical Study of Utility Functions in Multi-Objective Reinforcement Learning
Manel Rodriguez-Soto, Juan Antonio Rodriguez Aguilar, Maite López-Sánchez
Abstract
Multi-objective reinforcement learning (MORL) is an excellent framework for multi-objective sequential decision-making. MORL employs a utility function to aggregate multiple objectives into one that expresses a user's preferences. However, MORL still misses two crucial theoretical analyses of the properties of utility functions: (1) a characterisation of the utility functions for which an associated optimal policy exists, and (2) a characterisation of the types of preferences that can be expressed as utility functions. As a result, we formally characterise the families of preferences and utility functions that MORL should focus on: those for which an optimal policy is guaranteed to exist. We expect our theoretical results to promote the development of novel MORL algorithms that exploit our theoretical findings.
BibTeX
@inproceedings{
rodriguez-soto2024an,
title={An Analytical Study of Utility Functions in Multi-Objective Reinforcement Learning},
author={Manel Rodriguez-Soto and Juan Antonio Rodriguez Aguilar and Maite L{\'o}pez-S{\'a}nchez},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=K3h2kZFz8h}
}