ReFINE: A Reward-Based Framework for Interpretable and Nuanced Evaluation of Radiology Report Generation
Yunyi Liu, Yingshu Li, Zhanyu Wang, Xinyu Liang, Lingqiao Liu, Lei Wang, Luping Zhou
Abstract
Automated radiology report generation (R2Gen) has advanced significantly, yet evaluation remains challenging due to the complexity of assessing report quality. Traditional metrics often misalign with human judgments, failing to identify specific deficiencies. To address this, we introduce ReFINE, a framework for training an Evaluation Model using a novel margin-based reward enforcement loss. This approach decomposes report quality into fine-grained sub-scores across user-defined criteria, improving interpretability. Leveraging GPT-4, we generate diverse training data with paired accepted and rejected reports to train our model under a reward-based system. The trained ReFINE Score provides both granular sub-scores and an aggregated quality assessment, enabling criterion-specific evaluation. Experimental results demonstrate ReFINE
BibTeX
@inproceedings{aaai2026_refinearewardbas,
title = {ReFINE: A Reward-Based Framework for Interpretable and Nuanced Evaluation of Radiology Report Generation},
author = {Yunyi Liu and Yingshu Li and Zhanyu Wang and Xinyu Liang and Lingqiao Liu and Lei Wang and Luping Zhou},
booktitle = {AAAI 2026},
year = {2026}
}