2022
Revisiting Over-smoothing in BERT from the Perspective of Graph
ICLR 2022spotlight
Recently over-smoothing phenomenon of Transformer-based models is observed in both vision and language fields. However, no existing work has delved deeper to further investigate the main cause of this phenomenon. In this work, we make the attempt to analyze the over-smoothing problem from the perspe…