Ruonan Wu

Enhancing Self-Attention with Knowledge-Assisted Attention Maps

Large-scale pre-trained language models have attracted extensive attentions in the research community and shown promising results on various tasks of natural language processing. However, the attention maps, which record the attention scores between tokens in self-attention mechanism, are sometimes…