Discrete Latent Features Ablate Adversarial Attack: A Robust Prompt Tuning Framework for VLMs
While adversarial fine-tuning can enhance the robustness of vision-language models (VLMs), such approaches are computationally expensive. Adversarial prompt tuning has emerged as a practical alternative. However, existing methods are limited by their reliance on vulnerable continuous image features.…