2026
Query Lens: Interpreting Sparse Key-Value Features with Indirect Effects
ICML 2026poster
While sparse autoencoders yield features easier to study than individual neurons, their reliable interpretation remains challenging. We propose Query Lens, which extends Logit Lens to provide more comprehensive and faithful interpretations of sparse features. By jointly considering encoder-side key …