H-SPLID: HSIC-based Saliency Preserving Latent Information Decomposition
Lukas Miklautz, Chengzhi Shi, Andrii Shkabrii, Theodoros Thirimachos Davarakis, Prudence Lam, Claudia Plant, Jennifer Dy, Stratis Ioannidis
Abstract
We introduce H-SPLID, a novel algorithm for learning salient feature representations through the explicit decomposition of salient and non-salient features into separate spaces. We show that H-SPLID promotes learning low-dimensional, task-relevant features. We prove that the expected prediction deviation under input perturbations is upper-bounded by the dimension of the salient subspace and the Hilbert-Schmidt Independence Criterion (HSIC) between inputs and representations. This establishes a link between robustness and latent representation compression in terms of the dimensionality and information preserved. Empirical evaluations on image classification tasks show that models trained with H-SPLID primarily rely on salient input components, as indicated by reduced sensitivity to perturbations affecting non-salient features, such as image backgrounds.
BibTeX
@inproceedings{
miklautz2025hsplid,
title={H-{SPLID}: {HSIC}-based Saliency Preserving Latent Information Decomposition},
author={Lukas Miklautz and Chengzhi Shi and Andrii Shkabrii and Theodoros Thirimachos Davarakis and Prudence Lam and Claudia Plant and Jennifer Dy and Stratis Ioannidis},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=0owHzJrnB6}
}