Jimmy Chih-Hsien Peng

Breaking the Block: Preserving Data Continuity to Train Superior SAEs for Instruct Models

Sparse Autoencoders (SAEs) have become a cornerstone in mechanistic interpretability. However, current training methods inherit the Block Training paradigm from LLM pre-training. We identify this as a critical methodological oversight when applied to instruct models. Theoretically, utilizing GSNR an…