3 accepted papers
Comparing human and model performance offers a valuable perspective for understanding the strengths and limitations of embedding models, highlighting wherethey succeed and where they fail to capture meaning and nuance. However, such comparisons are rarely made, as human performance on embedding task…
Image representations are often evaluated through disjointed, task-specific protocols, leading to a fragmented understanding of model capabilities. For instance, it is unclear whether an image embedding model adept at clustering images is equally good at retrieving relevant images given a piece of t…
Text embeddings are typically evaluated on a narrow set of tasks, limited in terms of languages, domains, and task types. To circumvent this limitation and to provide a more comprehensive evaluation, we introduce the Massive Multilingual Text Embedding Benchmark (MMTEB) -- a large-scale community-dr…