Sofia Lee

Impact of Task Adapting on Transformer Models for Targeted Sentiment Analysis in Croatian Headlines

Transformer models, such as BERT, are often taken off-the-shelf and then fine-tuned on a downstream task. Although this is sufficient for many tasks, low-resource settings require special attention. We demonstrate an approach of performing an extra stage of self-supervised task-adaptive pre-training…