Dmitrii Kharchev

NB-MLM: Efficient Domain Adaptation of Masked Language Models for Sentiment Analysis

While Masked Language Models (MLM) are pre-trained on massive datasets, the additional training with the MLM objective on domain or task-specific data before fine-tuning for the final task is known to improve the final performance. This is usually referred to as the domain or task adaptation step. H…