Gu Simiu

UEGP: Unified Expert-Guided Pre-training for Knowledge Rekindle

Pre-training and fine-tuning framework has become the standard training paradigm for NLP tasks and is also widely used in industrial-level applications. However, there are still a limitation with this paradigm: simply fine-tuning with task-specific objectives tends to converge to local minima, resul…