Sparse Activation Editing for Reliable Instruction Following in Narratives
Runcong Zhao, Chengyu Cao, Qinglin Zhu, Xiucheng Ly, Shun Shao, Lin Gui, Ruifeng Xu, Yulan He
Abstract
Complex narrative contexts often challenge language models’ ability to follow instructions, and existing benchmarks fail to capture these difficulties. To address this, we propose Concise-SAE, a training-free framework that improves instruction following by identifying and editing instruction-relevant neurons using only natural language instructions, without requiring labelled data. To thoroughly evaluate our method, we introduce FreeInstruct, a diverse and realistic benchmark that highlights the challenges of instruction following in narrative-rich settings. While initially motivated by complex narratives, Concise-SAE demonstrates state-of-the-art instruction adherence across varied tasks without compromising generation quality.
BibTeX
@inproceedings{emnlp2025_sparseactivation,
title = {Sparse Activation Editing for Reliable Instruction Following in Narratives},
author = {Runcong Zhao and Chengyu Cao and Qinglin Zhu and Xiucheng Ly and Shun Shao and Lin Gui and Ruifeng Xu and Yulan He},
booktitle = {EMNLP 2025},
year = {2025}
}