2025
A Good Plan is Hard to Find: Aligning Models with Preferences is Misaligned with What Helps Users
EMNLP 2025
To assist users in complex tasks, LLMs generate plans: step-by-step instructions towards a goal. While alignment methods aim to ensure LLM plans are helpful, they train (RLHF) or evaluate (ChatbotArena) on what users prefer, assuming this reflects what helps them. We test this with Planorama: an int