HCoTT: Hierarchical Chain-of-Thought Distillation

Abstract

Chains of Thought (CoT) have shown potential in augmenting the reasoning capabilities of language models, yet their effectiveness is predominantly observed in large language models (LLMs). Recently, several attempts have been made to inject CoT into small language models (SLMs) using distillation and achieved promising results. However, current methods (1) ignore the rationality and hierarchical logic of reasoning when constructing CoT; (2) fail to inject hierarchical reasoning priors into SLMs. In this paper, we design a Hierarchical CoT distillation framework termed HCoTT, whose core component is a hierarchical recursive sampling module and a hierarchical learning module. Specifically, hierarchical recursive sampling utilizes a hierarchical logic process to generate more diverse explanations and a Hierarchical Chain of Thought (HCoT). Furthermore, hierarchical learning encompasses hierarchical supervision and representation learning, which is designed to augment learning and representation of implicit explanatory priors in HCoT for SLMs. Experimental results show that HCoTT can effectively improve the performance of SLMs on Faculty-Reasoning and Multiple-Choice QA tasks. More impressively, our method is model-independent and can consistently improve performance with existing language model fusions of different scales.