Stabilizing Cross-Modal Bidirectional Attribution: Few-Shot Adversarial Prompt Tuning for Robust Vision-Language Models

Jun Feng; Shuhong Wu; Hong Sun; Pengfei Zhang; Bocheng Ren; Shunli Zhang

AAAI 2026technical0 citations

Stabilizing Cross-Modal Bidirectional Attribution: Few-Shot Adversarial Prompt Tuning for Robust Vision-Language Models

Jun Feng, Shuhong Wu, Hong Sun, Pengfei Zhang, Bocheng Ren, Shunli Zhang

Abstract

Large-scale pre-trained vision-language models (VLMs) like CLIP show exceptional performance and zero-shot generalization. However, their reliability may be severely undermined by a critical vulnerability to subtle adversarial perturbations. Our work reveals a critical cross-modal vulnerability: visual-only perturbations induce substantial, synchronous shifts in decision attribution maps across both image and text. This phenomenon signifies a fundamental disruption of the VLM

BibTeX

@inproceedings{aaai2026_stabilizingcross,
  title = {Stabilizing Cross-Modal Bidirectional Attribution: Few-Shot Adversarial Prompt Tuning for Robust Vision-Language Models},
  author = {Jun Feng and Shuhong Wu and Hong Sun and Pengfei Zhang and Bocheng Ren and Shunli Zhang},
  booktitle = {AAAI 2026},
  year = {2026}
}