← Search

Jan Šnajder

2 accepted papers

2024

Are ELECTRA’s Sentence Embeddings Beyond Repair? The Case of Semantic Textual Similarity

EMNLP 2024finding

While BERT produces high-quality sentence embeddings, its pre-training computational cost is a significant drawback. In contrast, ELECTRA provides a cost-effective pre-training objective and downstream task performance improvements, but worse sentence embeddings. The community tacitly stopped utiliz…