Ariana Azarbal

Recontextualization Mitigates Specification Gaming Without Modifying the Specification

Developers often struggle to specify correct training labels and rewards. Perhaps they don't need to. We propose recontextualization, which reduces how often language models "game" training signals, performing misbehaviors those signals fail to penalize. We show recontextualization prevents models f…