2023
PEER: Pre-training ELECTRA Extended by Ranking
ACL 2023findings
The BERT model and its variants have made great achievements in many downstream natural language processing tasks. The achievements of these models, however, demand highly expensive pre-training computation cost. To address this pre-training efficiency issue, the ELECTRA model is proposed to use a d…