Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations
Mikołaj Sacha, Bartosz Jura, Dawid Rymarczyk, Łukasz Struski, Jacek Tabor, Bartosz Zieliński
Abstract
Prototypical parts-based networks are becoming increasingly popular due to their faithful self-explanations. However, their similarity maps are calculated in the penultimate network layer. Therefore, the receptive field of the prototype activation region often depends on parts of the image outside this region, which can lead to misleading interpretations. We name this undesired behavior a spatial explanation misalignment and introduce an interpretability benchmark with a set of dedicated metrics for quantifying this phenomenon. In addition, we propose a method for misalignment compensation and apply it to existing state-of-the-art models. We show the expressiveness of our benchmark and the effectiveness of the proposed compensation methodology through extensive empirical studies.
BibTeX
@article{Sacha_Jura_Rymarczyk_Struski_Tabor_Zieliński_2024, title={Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations}, volume={38}, url={https://ojs.aaai.org/index.php/AAAI/article/view/30154}, DOI={10.1609/aaai.v38i19.30154}, abstractNote={Prototypical parts-based networks are becoming increasingly popular due to their faithful self-explanations. However, their similarity maps are calculated in the penultimate network layer. Therefore, the receptive field of the prototype activation region often depends on parts of the image outside this region, which can lead to misleading interpretations. We name this undesired behavior a spatial explanation misalignment and introduce an interpretability benchmark with a set of dedicated metrics for quantifying this phenomenon. In addition, we propose a method for misalignment compensation and apply it to existing state-of-the-art models. We show the expressiveness of our benchmark and the effectiveness of the proposed compensation methodology through extensive empirical studies.}, number={19}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Sacha, Mikołaj and Jura, Bartosz and Rymarczyk, Dawid and Struski, Łukasz and Tabor, Jacek and Zieliński, Bartosz}, year={2024}, month={Mar.}, pages={21563-21573} }