Enhancing Chain-of-Thought Reasoning via Neuron Activation Differential Analysis
Yiru Tang, Kun Zhou, Yingqian Min, Wayne Xin Zhao, Jing Sha, Zhichao Sheng, Shijin Wang
Abstract
Despite the impressive chain-of-thought(CoT) reasoning ability of large language models (LLMs), its underlying mechanisms remains unclear. In this paper, we explore the inner workings of LLM’s CoT ability via the lens of neurons in the feed-forward layers. We propose an efficient method to identify reasoning-critical neurons by analyzing their activation patterns under reasoning chains of varying quality. Based on it, we devise a rather simple intervention method that directly stimulates these reasoning-critical neurons, to guide the generation of high-quality reasoning chains. Extended experiments validate the effectiveness of our method and demonstrate the critical role these identified neurons play in CoT reasoning.
BibTeX
@inproceedings{emnlp2025_enhancingchainof,
title = {Enhancing Chain-of-Thought Reasoning via Neuron Activation Differential Analysis},
author = {Yiru Tang and Kun Zhou and Yingqian Min and Wayne Xin Zhao and Jing Sha and Zhichao Sheng and Shijin Wang},
booktitle = {EMNLP 2025},
year = {2025}
}