Lookahead-GCG: Improving Multi-Model Gradient-Based Jailbreaking Attacks via Nesterov Momentum
Transferable jailbreaking attacks enable red-teaming of black-box large language models by optimizing adversarial prompts on open-source surrogates. A natural approach to improve transferability is multi-model training---optimizing against multiple source models simultaneously. Yet this approach has…