Adaptive Curriculum Learning With Successor Features for Imbalanced Compositional Reward Functions

Abstract

This work addresses the challenge of reinforcement learning with reward functions that feature highly imbalanced components in terms of importance and scale. Reinforcement learning algorithms generally struggle to handle such imbalanced reward functions effectively. Consequently, they often converge to suboptimal policies that favor only the dominant reward component. For example, agents might adopt passive strategies, avoiding any action to evade potentially unsafe outcomes entirely. To mitigate the adverse effects of imbalanced reward functions, we introduce a curriculum learning approach based on the successor features representation. This novel approach enables our learning system to acquire policies that take into account all reward components, allowing for a more balanced and versatile decision-making process.