Regression Aware Inference with LLMs

Michal Lukasik; Harikrishna Narasimhan; Aditya Krishna Menon; Felix Yu; Sanjiv Kumar

EMNLP 2024finding3 citations

Regression Aware Inference with LLMs

Michal Lukasik, Harikrishna Narasimhan, Aditya Krishna Menon, Felix Yu, Sanjiv Kumar

Abstract

Large language models (LLMs) have shown strong results on a range of applications, including regression and scoring tasks.Typically, one obtains outputs from an LLM via autoregressive sampling from the model’s output distribution. We show that this inference strategy can be sub-optimal for common regression and scoring evaluation metrics. As a remedy, we build on prior work on Minimum Bayes Risk decoding,and propose alternate inference strategies that estimate the Bayes-optimal solution for regression and scoring metrics in closed-form from sampled responses.We show that our proposal significantly improves over baselines across datasets and models.

BibTeX

@inproceedings{lukasik-etal-2024-regression,
    title = "Regression Aware Inference with {LLM}s",
    author = "Lukasik, Michal  and
      Narasimhan, Harikrishna  and
      Menon, Aditya Krishna  and
      Yu, Felix  and
      Kumar, Sanjiv",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.findings-emnlp.799/",
    doi = "10.18653/v1/2024.findings-emnlp.799",
    pages = "13667--13678"
}