ProvBench: A Benchmark of Legal Provision Recommendation for Contract Auto-Reviewing
Xiuxuan Shen, Zhongyuan Jiang, Junsan Zhang, Junxiao Han, Yao Wan, Chengjie Guo, Bingcheng Liu, Jie Wu
Abstract
Contract review is a critical process to protect the rights and interests of the parties involved. However, this process is time-consuming, labor-intensive, and costly, especially when a contract faces multiple rounds of review. To accelerate the contract review and promote the completion of transactions, this paper introduces a novel benchmark of legal provision recommendation and conflict detection for contract auto-reviewing (ProvBench), which aims to recommend the legal provisions related to contract clauses and detect possible legal conflicts. Specifically, we construct the first Legal Provision Recommendation Dataset: ProvData, which covers 8 common contract types. In addition, we conduct extensive experiments to evaluate ProvBench on various state-of-the-art models. Experimental results validate the feasibility of ProvBench and demonstrate the effectiveness of ProvData. Finally, we identify potential challenges in the ProvBench and advocate for further investigation.
BibTeX
@inproceedings{shen-etal-2025-provbench,
title = "{P}rov{B}ench: A Benchmark of Legal Provision Recommendation for Contract Auto-Reviewing",
author = "Shen, Xiuxuan and
Jiang, Zhongyuan and
Zhang, Junsan and
Han, Junxiao and
Wan, Yao and
Guo, Chengjie and
Liu, Bingcheng and
Wu, Jie and
Li, Renxiang and
Yu, Philip S.",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.acl-long.312/",
doi = "10.18653/v1/2025.acl-long.312",
pages = "6240--6254",
ISBN = "979-8-89176-251-0"
}