Angelo Pimienta

A Novel Approach to Evaluating Evaluation Metrics for Multi-Output Structured Prediction

In multi-output structured prediction tasks, while only one ground truth label may be provided in the training data, multiple equally valid outputs may be possible, making reliable evaluation a persistent challenge. We postulate that human evaluators implicitly use task-specific invariants, e.g., ob