Angelo Basile

2 accepted papers

2024

PyRater: A Python Toolkit for Annotation Analysis

Angelo Basile, Marc Franco-Salvador, Paolo Rosso

COLING 2024main

We introduce PyRater, an open-source Python toolkit designed for analysing corpora annotations. When creating new annotated language resources, probabilistic models of annotation are the state-of-the-art solution for identifying the best annotators, retrieving the gold standard, and more generally s…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗

2023

Zero-Shot Data Maps. Efficient Dataset Cartography Without Model Training

Angelo Basile, Marc Franco-Salvador, Paolo Rosso

EMNLP 2023long findings

Data Maps (Swayamdipta, et al. 2020) have emerged as a powerful tool for diagnosing large annotated datasets. Given a model fitted on a dataset, these maps show each data instance from the dataset in a 2-dimensional space defined by a) the model's confidence in the true class and b) the variability…

Cited by 0 Source ↗code ↗Scholar ↗