Jiyang Shen

MoDr: Mixture-of-Depth-Recurrent Transformers for Test-Time Reasoning

Large Language Models have demonstrated superior reasoning capabilities by generating step-by-step reasoning in natural language before deriving the final answer. Recently, Geiping et al. introduced 3.5B-Huginn as an alternative to this paradigm, a depth-recurrent Transformer that increases computat…