Longjie Cui

Towards Database-Free Text-to-SQL Evaluation: A Graph-Based Metric for Functional Correctness

Execution Accuracy and Exact Set Match are two predominant metrics for evaluating the functional correctness of SQL queries in modern Text-to-SQL tasks. However, both metrics have notable limitations: Exact Set Match fails when queries are functionally equivalent but syntactically different, while E…