Iuri Macocco — accepted AI-conference papers

2025

Emergence of a High-Dimensional Abstraction Phase in Language Transformers

Emily Cheng, Diego Doimo, Corentin Kervadec, Iuri Macocco, Lei Yu, Alessandro Laio +1

ICLR 2025poster

A language model (LM) is a mapping from a linguistic context to an output token. However, much remains to be known about this mapping, including how its geometric properties relate to its function. We take a high-level geometric approach to its analysis, observing, across five pre-trained transforme…

Cited by 9 Source ↗PDF ↗code ↗Scholar ↗

2025

Prediction Hubs are Context-Informed Frequent Tokens in LLMs

Beatrix Miranda Ginn Nielsen, Iuri Macocco, Marco Baroni

ACL 2025long

Hubness, the tendency for a few points to be among the nearest neighbours of a disproportionate number of other points, commonly arises when applying standard distance measures to high-dimensional data, often negatively impacting distance-based analysis. As autoregressive large language models (LLMs…

Cited by 0 Source ↗PDF ↗Scholar ↗