2026
PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space
ICML 2026spotlight
The remarkable success of Chain-of-Thought (CoT), which enhances performance by scaling generation steps at test-time, inspires us to ask: can we leverage a similar scaling of computational steps during pretraining to improve the generation of each individual token? To address this, we propose a nov…