← Search

Subhrajit Roy

3 accepted papers

2025

Safety Alignment Should be Made More Than Just a Few Tokens Deep

ICLR 2025oral

The safety alignment of current Large Language Models (LLMs) is vulnerable. Simple attacks, or even benign fine-tuning, can jailbreak aligned models. We note that many of these vulnerabilities are related to a shared underlying issue: safety alignment can take shortcuts, wherein the alignment adapts…

2024

Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering

ICLR 2024poster

Prompting and in-context learning (ICL) have become efficient learning paradigms for large language models (LLMs). However, LLMs suffer from prompt brittleness and various bias factors in the prompt, including but not limited to the formatting, the choice verbalizers, and the ICL examples. To addres…

Cited by 51SourcePDFScholar
2022

Diagnosing failures of fairness transfer across distribution shift in real-world medical settings

NeurIPS 2022accept

Diagnosing and mitigating changes in model fairness under distribution shift is an important component of the safe deployment of machine learning in healthcare settings. Importantly, the success of any mitigation strategy strongly depends on the \textit{structure} of the shift. Despite this, there h…

Cited by 74SourcePDFScholar