Wang Lijie

Syntax-guided Contrastive Learning for Pre-trained Language Model

Syntactic information has been proved to be useful for transformer-based pre-trained language models. Previous studies often rely on additional syntax-guided attention components to enhance the transformer, which require more parameters and additional syntactic parsing in downstream tasks. This incr…