Jiangxia Cao

Enhancing Chinese Pre-trained Language Model via Heterogeneous Linguistics Graph

Chinese pre-trained language models usually exploit contextual character information to learn representations, while ignoring the linguistics knowledge, e.g., word and sentence information. Hence, we propose a task-free enhancement module termed as Heterogeneous Linguistics Graph (HLG) to enhance Ch…