Relevance Scores Calibration for Ranked List Truncation via TMP Adapter
Pavel Posokhov, Sergei Masliukhin, Skrylnikov Stepan, Danil Tirskikh, Olesia Makhnytkina
Abstract
The ranked list truncation task involves determining a truncation point to retrieve the relevant items from a ranked list. Despite current advancements, truncation methods struggle with limited capacity, unstable training and inconsistency of selected threshold. To address these problems we introduce TMP Adapter, a novel approach that builds upon the improved adapter model and incorporates the Threshold Margin Penalty (TMP) as an additive loss function to calibrate ranking model relevance scores for ranked list truncation. We evaluate TMP Adapter’s performance on various retrieval datasets and observe that TMP Adapter is a promising advancement in the calibration methods, which offers both theoretical and practical benefits for ranked list truncation.
BibTeX
@inproceedings{posokhov-etal-2025-relevance,
title = "Relevance Scores Calibration for Ranked List Truncation via {TMP} Adapter",
author = "Posokhov, Pavel and
Masliukhin, Sergei and
Stepan, Skrylnikov and
Tirskikh, Danil and
Makhnytkina, Olesia",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-acl.402/",
doi = "10.18653/v1/2025.findings-acl.402",
pages = "7728--7734",
ISBN = "979-8-89176-256-5"
}