Activation Manipulation Attack: Penetrating and Harmful Jailbreak Attack Against Large Vision-Language Models
Haojie Hao, Jiakai Wang, Aishan Liu, Yuqing Ma, Haotong Qin, Yuanfang Guo, Xianglong Liu
Abstract
Recently, Large Vision-Language Models (LVLMs) have been demonstrated to be vulnerable to jailbreak attacks, highlighting the urgent need for further research to comprehensively identify and mitigate these threats. Unfortunately, existing jailbreak studies primarily focus on coarse-grained input manipulation to elicit specific responses, overlooking the exploitation of internal representations, i.e., intermediate activations, which constrains their ability to penetrate alignment safeguards and generate harmful responses. To tackle this issue, we propose the Activation Manipulation (ActMan) Attack framework, which performs fine-grained activation manipulations inspired by the perception and cognition stages of human decision-making, enhancing both the penetration capability and harmfulness of attacks. To improve penetration capability, we introduce a Deceptive Visual Camouflage module inspired by the masking effect in human perception. This module uses a benign activation-guided attention redirection strategy to conceal abnormal activation patterns, thereby suppressing LVLM
BibTeX
@inproceedings{aaai2026_activationmanipu,
title = {Activation Manipulation Attack: Penetrating and Harmful Jailbreak Attack Against Large Vision-Language Models},
author = {Haojie Hao and Jiakai Wang and Aishan Liu and Yuqing Ma and Haotong Qin and Yuanfang Guo and Xianglong Liu},
booktitle = {AAAI 2026},
year = {2026}
}