Search-Adaptor: Embedding Customization for Information Retrieval
Jinsung Yoon, Yanfei Chen, Sercan Arik, Tomas Pfister
Abstract
Embeddings extracted by pre-trained Large Language Models (LLMs) have significant potential to improve information retrieval and search. Beyond the zero-shot setup in which they are being conventionally used, being able to take advantage of the information from the relevant query-corpus paired data can further boost the LLM capabilities. In this paper, we propose a novel method, Search-Adaptor, for customizing LLMs for information retrieval in an efficient and robust way. Search-Adaptor modifies the embeddings generated by pre-trained LLMs, and can be integrated with any LLM, including those only available via prediction APIs. On multiple English, multilingual, and multimodal retrieval datasets, we show consistent and significant performance benefits for Search-Adaptor – e.g., more than 5% improvements for Google Embedding APIs in nDCG@10 averaged over 14 BEIR datasets.
BibTeX
@inproceedings{yoon-etal-2024-search,
title = "Search-Adaptor: Embedding Customization for Information Retrieval",
author = "Yoon, Jinsung and
Chen, Yanfei and
Arik, Sercan and
Pfister, Tomas",
editor = "Ku, Lun-Wei and
Martins, Andre and
Srikumar, Vivek",
booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = aug,
year = "2024",
address = "Bangkok, Thailand",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.acl-long.661/",
doi = "10.18653/v1/2024.acl-long.661",
pages = "12230--12247"
}