Mahesh Pasupuleti

The Alignment Waltz: Jointly Training Agents to Collaborate for Safety

Harnessing the power of LLMs requires a delicate dance between being helpful and harmless, leading to two critical challenges: vulnerability to adversarial attacks that elicit unsafe content, and a tendency for overrefusal on benign but sensitive prompts. Current approaches often navigate this dance…