EMNLP 2023short main0 citations
Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications
Manuel Faysse, Gautier Viaud, CELINE HUDELOT, Pierre Colombo
Abstract
Instruction Fine-Tuning (IFT) is a powerful paradigm that strengthens the zero-shot capabilities of Large Language Models (LLMs), but in doing so induces new evaluation metric requirements. We show LLM-based metrics to be well adapted to these requirements, and leverage them to conduct an investigation of task-specialization strategies, quantifying the trade-offs that emerge in practical industrial settings. Our findings offer practitioners actionable insights for real-world IFT model deployment.
Instruction FinetuningEvaluation MetricsLarge Language Models
BibTeX
@inproceedings{
faysse2023revisiting,
title={Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications},
author={Manuel Faysse and Gautier Viaud and CELINE HUDELOT and Pierre Colombo},
booktitle={The 2023 Conference on Empirical Methods in Natural Language Processing},
year={2023},
url={https://openreview.net/forum?id=Ror9xJhbdc}
}