Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought
Yu Huo, Siyu Zhang, Zeng Kun, Haoyue Liu, Owen Lee, Junlin chen, Lu YuQuan, Yifu Guo
Abstract
Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints—notably generative numeracy, attribute binding, and part-level relations. To address these challenges, we propose **Shape-of-Thought (SoT)**, a visual CoT framework that enables *progressive shape assembly represented as coherent 2D projections* without external engines at inference time. SoT trains a unified multimodal autoregressive model to generate interleaved textual plans and rendered intermediate states, helping the model capture shape-assembly logic without producing explicit geometric representations. To support this paradigm, we introduce **SoT-26K**, a large-scale dataset of grounded assembly traces derived from part-based CAD hierarchies, and **T2S-CompBench**, a benchmark for evaluating structural integrity and trace faithfulness. Fine-tuning on SoT-26K achieves 88.4\% on component numeracy and 84.8\% on structural topology, outperforming text-only baselines by around 20\%. SoT establishes a new paradigm for transparent, process-supervised compositional generation. The code is available at https://anonymous.4open.science/r/16FE/.
BibTeX
@inproceedings{
huo2026shape,
title={Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought},
author={Yu Huo and Siyu Zhang and Kun Zeng and Haoyue Liu and Owen Lee and Junlin chen and YUQUAN LU and Yifu Guo and Yaodong Liang and Xiaoying Tang},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=ZsJnkEgh6I}
}