2025
CARMO: Dynamic Criteria Generation for Context Aware Reward Modelling
ACL 2025finding
Reward modeling in large language models is known to be susceptible to reward hacking, causing models to latch onto superficial features such as the tendency to generate lists or unnecessarily long responses. In RLHF, and more generally during post-training, flawed reward signals often lead to outpu…