NeurIPS 2025poster0 citations
Antidistillation Sampling
Yash Savani, Asher Trockman, Zhili Feng, Yixuan Even Xu, Avi Schwarzschild, Alexander Robey, Marc Anton Finzi, J Zico Kolter
Abstract
Frontier models that generate extended reasoning traces inadvertently produce token sequences that can facilitate model distillation. Recognizing this vulnerability, model owners may seek sampling strategies that limit the effectiveness of distillation without compromising model performance. *Antidistillation sampling* provides exactly this capability. By strategically modifying a model's next-token probability distribution, antidistillation sampling poisons reasoning traces, rendering them significantly less effective for distillation while preserving the model's utility.
LLMdistillationsecurity
BibTeX
@inproceedings{
savani2025antidistillation,
title={Antidistillation Sampling},
author={Yash Savani and Asher Trockman and Zhili Feng and Yixuan Even Xu and Avi Schwarzschild and Alexander Robey and Marc Anton Finzi and J Zico Kolter},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=Vo2UHqMu8t}
}