On Pruning State-Space LLMs

Tamer Ghattas; Michael Hassid; Roy Schwartz

EMNLP 20250 citations

On Pruning State-Space LLMs

Tamer Ghattas, Michael Hassid, Roy Schwartz

Abstract

Recent work proposed state-space models (SSMs) as an efficient alternative to transformer-based LLMs. Can these models be pruned to further reduce their computation costs? We adapt several pruning methods to the SSM structure, and apply them to four SSM-based LLMs across multiple tasks. We find that such models are quite robust to some pruning methods (e.g., WANDA), while using other methods lead to fast performance degradation.

BibTeX

@inproceedings{emnlp2025_onpruningstatesp,
  title = {On Pruning State-Space LLMs},
  author = {Tamer Ghattas and Michael Hassid and Roy Schwartz},
  booktitle = {EMNLP 2025},
  year = {2025}
}