SGM: A Dataset for 3D Garment Reconstruction from Single Hand-Drawn Sketch

Abstract

High-fidelity garment reconstruction is essential for various applications such as garment design and virtual try-on. While image-based reconstruction methods have made significant progress with deep generative models, generating 3D models from hand-drawn sketches to meet design intentions remains challenging. One of the main obstacles is the limited availability of large-scale 3D garment models accompanied by corresponding sketches. To address this issue, we propose SGM, a comprehensive dataset comprising 656 garment models categorized into short and long sleeves. Each garment model in SGM is accompanied by four types of rendered images and a series of UDF values. Furthermore, we introduce a novel baseline approach for sketch-based garment reconstruction using an end-to-end generative network capable of generating garment models from single hand-drawn sketches. Extensive experimental results highlight the significance and value of our proposed dataset and method. We plan to make SGM publicly available upon publication.