An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4
Hui Huang, Xingyuan Bu, Hongli Zhou, Yingqi Qu, Jing Liu, Muyun Yang, Bing Xu, Tiejun Zhao
Abstract
Recently, there has been a growing trend of utilizing Large Language Model (LLM) to evaluate the quality of other LLMs. Many studies have fine-tuned judge models based on open-source LLMs for evaluation. While the fine-tuned judge models are claimed to achieve comparable evaluation capability with GPT-4, in this work, we conduct an empirical study of LLM-as-a-Judge. Our findings indicate that although the fine-tuned judge models achieve high performance on in-domain test sets, even surpassing GPT-4, they underperform GPT-4 across several dimensions, including generalizability, fairness and adaptability. We also reveal that the fine-tuned judge model inherently operates as a task-specific classifier, consequently imposing the limitations.
BibTeX
@inproceedings{huang-etal-2025-empirical,
title = "An Empirical Study of {LLM}-as-a-Judge for {LLM} Evaluation: Fine-tuned Judge Model is not a General Substitute for {GPT}-4",
author = "Huang, Hui and
Bu, Xingyuan and
Zhou, Hongli and
Qu, Yingqi and
Liu, Jing and
Yang, Muyun and
Xu, Bing and
Zhao, Tiejun",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-acl.306/",
doi = "10.18653/v1/2025.findings-acl.306",
pages = "5880--5895",
ISBN = "979-8-89176-256-5"
}