Pavel Kalinin

Batch-Softmax Contrastive Loss for Pairwise Sentence Scoring Tasks

The use of contrastive loss for representation learning has become prominent in computer vision, and it is now getting attention in Natural Language Processing (NLP).Here, we explore the idea of using a batch-softmax contrastive loss when fine-tuning large-scale pre-trained transformer models to lea…