Grid Distillation: Compositional Image Distillation via Structured Generative Grids

Abstract

We present Grid Distillation, a generative dataset distillation framework that compresses large-scale datasets into a compact set of informative synthetic samples. Our method constructs high-resolution compositional grids via spectral submodular optimization, which injects world knowledge from CLIP representations to maximize semantic coverage and diversity. These grids are then downsampled into low-resolution distilled images optimized for diversity and representational efficiency. During training, a single-step diffusion reconstruction (based on Stable Diffusion Turbo) restores fine-grained spatial details from diffusion priors, bridging the gap between compact representations and natural image statistics. A grid-aware cropping strategy further enhances discriminability by probabilistically aligning crops with grid boundaries, maintaining compatibility with standard 224 x 224 inference inputs. Experiments on ImageWoof, ImageNette, ImageIDC, and ImageNet-1K demonstrate consistent improvements over existing dataset distillation methods across multiple IPC settings.