← Search

Tingyang Yu

2 accepted papers

2024

Fool Your (Vision and) Language Model with Embarrassingly Simple Permutations

ICML 2024poster

Large language and vision-language models are rapidly being deployed in practice thanks to their impressive capabilities in instruction following, in-context learning, and so on. This raises an urgent need to carefully analyse their robustness so that stakeholders can understand if and when such mod…

2024

Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models

ICML 2024poster

Current vision large language models (VLLMs) exhibit remarkable capabilities yet are prone to generate harmful content and are vulnerable to even the simplest jailbreaking attacks. Our initial analysis finds that this is due to the presence of harmful data during vision-language instruction fine-tun…