TIDE: Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation
Victor Shea-Jay Huang, Le Zhuo, Yi Xin, Zhaokai Wang, Fu-Yun Wang, Yuchi Wang, Renrui Zhang, Peng Gao
Abstract
Diffusion Transformers (DiTs) are a powerful yet underexplored class of generative models compared to U-Net-based diffusion architectures. We propose TIDE—Temporal-aware sparse autoencoders for Interpretable Diffusion transformErs—a framework designed to extract sparse, interpretable activation features across timesteps in DiTs. TIDE effectively captures temporally-varying representations and reveals that DiTs naturally learn hierarchical semantics (e.g., 3D structure, object class, and fine-grained concepts) during large-scale pretraining. Experiments show that TIDE enhances interpretability and controllability while maintaining reasonable generation quality, enabling applications such as safe image editing and style transfer.
BibTeX
@inproceedings{aaai2026_tidetemporalawar,
title = {TIDE: Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation},
author = {Victor Shea-Jay Huang and Le Zhuo and Yi Xin and Zhaokai Wang and Fu-Yun Wang and Yuchi Wang and Renrui Zhang and Peng Gao and Hongsheng Li},
booktitle = {AAAI 2026},
year = {2026}
}