Kamel Smaïli

Supervised Contrastive Learning as Multi-Objective Optimization for Fine-Tuning Large Pre-Trained Language Models

Recently, Supervised Contrastive Learning (SCL) has been shown to significantly outperform the well-known cross-entropy loss-based learning on most classification tasks. In SCL, a neural network is trained to optimize two objectives: pull an anchor and positive samples together in the embedding spac…