The Impact of Decorrelation on Transformer Interpretation Methods: Applications to Clinical Speech AI

Abstract

Recent applications of decorrelation methods to the multi-head attention layers and output embeddings of transformer-based models have resulted in improvements in efficiency and accuracy. Despite these advancements, there is a lack of research focused on the influence of decorrelation on transformer interpretation techniques. This study investigated the impact of two decorrelation methods on interpreting the decision-making logic of a Bidirectional Encoder Representations from Transformers (BERT) model. Two metrics, namely Comprehensiveness and Sufficiency, were used to quantify the interpretation quality, while the changes in correlation within each multi-head self-attention layer was statistically analyzed. Results indicate that decorrelating BERT embeddings leads to a sparser distribution of weights in the middle attention layers and a significantly improved interpretation quality. Conversely, decorrelating the attention maps of specific attention layers increases the correlation in the corresponding attention weight matrices, yielding a less marked improvement in interpretation quality and, in some instances, degraded model performance.