AAAI 2026technical0 citations
Outlier Matters: Efficient Long-to-Short Reasoning via Outlier-Guided Model Merging
Qiyuan Zhu, Dezhi Li, Lujun Li, Xiaoyu Qin, Wei Li, Hao Gu, Hua Xu, Sirui Han
Abstract
Large Reasoning Language Models (LRMs) have recently shown remarkable performance in complex reasoning tasks, but their extensive reasoning chains incur substantial computational overhead. To address this challenge, we propose Outlier-aware Reasoning Conciseness Adaptive Merge (ORCA), a novel plug-and-play model merging framework that leverages outlier activation patterns to fuse base models with reasoning models. Our ORCA introduces three key innovations: (1) adaptive alignment that reduces conflicts between disparate activation patterns during merging, (2) outlier-guided allocation that assigns merging coefficients proportional to each layer
BibTeX
@inproceedings{aaai2026_outliermattersef,
title = {Outlier Matters: Efficient Long-to-Short Reasoning via Outlier-Guided Model Merging},
author = {Qiyuan Zhu and Dezhi Li and Lujun Li and Xiaoyu Qin and Wei Li and Hao Gu and Hua Xu and Sirui Han and Yike Guo},
booktitle = {AAAI 2026},
year = {2026}
}