LLM Stinger: Jailbreaking LLMs Using RL Fine-Tuned LLMs (Student Abstract)
Piyush Jha, Arnav Arora, Vijay Ganesh
Abstract
We introduce LLM Stinger, a novel approach that leverages Large Language Models (LLMs) to automatically generate adversarial suffixes for jailbreak attacks. Unlike traditional methods, which require complex prompt engineering or white-box access, LLM Stinger uses a reinforcement learning (RL) loop to fine-tune an attacker LLM, generating new suffixes based on existing attacks for harmful questions from the HarmBench benchmark. Our method significantly outperforms existing red-teaming approaches (we compared against 15 of the latest methods), achieving a +57.2% improvement in Attack Success Rate (ASR) on LLaMA2-7B-chat and a +50.3% ASR increase on Claude 2, both models known for their extensive safety measures. Additionally, we achieved a 94.97% ASR on GPT-3.5 and 99.4% on Gemma-2B-it, demonstrating the robustness and adaptability of LLM Stinger across open and closed-source models.
BibTeX
@article{Jha_Arora_Ganesh_2025, title={LLM Stinger: Jailbreaking LLMs Using RL Fine-Tuned LLMs (Student Abstract)}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/35263}, DOI={10.1609/aaai.v39i28.35263}, abstractNote={We introduce LLM Stinger, a novel approach that leverages Large Language Models (LLMs) to automatically generate adversarial suffixes for jailbreak attacks. Unlike traditional methods, which require complex prompt engineering or white-box access, LLM Stinger uses a reinforcement learning (RL) loop to fine-tune an attacker LLM, generating new suffixes based on existing attacks for harmful questions from the HarmBench benchmark. Our method significantly outperforms existing red-teaming approaches (we compared against 15 of the latest methods), achieving a +57.2% improvement in Attack Success Rate (ASR) on LLaMA2-7B-chat and a +50.3% ASR increase on Claude 2, both models known for their extensive safety measures. Additionally, we achieved a 94.97% ASR on GPT-3.5 and 99.4% on Gemma-2B-it, demonstrating the robustness and adaptability of LLM Stinger across open and closed-source models.}, number={28}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Jha, Piyush and Arora, Arnav and Ganesh, Vijay}, year={2025}, month={Apr.}, pages={29393-29395} }