2026
Rethinking Thinking Tokens: LLMs as Improvement Operators
ICML 2026poster
Reasoning training incentivizes LLMs to produce long chains of thought (long CoT), which among other things, allows them to explore solution strategies with self-checking. This results in higher accuracy, but inflates context length, token/compute cost, and answer latency. We ask: Can current models…