2022
Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model
EMNLP 2022main
Pre-trained multilingual language models play an important role in cross-lingual natural language understanding tasks. However, existing methods did not focus on learning the semantic structure of representation, and thus could not optimize their performance. In this paper, we propose Multi-level Mu…