AIConfPaper
← Search

Raffaella Bernardi

9 accepted papers

2025

All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark

Davide Testa, Giovanni Bonetta, Raffaella Bernardi, Alessandro Bondielli, Alessandro Lenci, Alessio Miaschi +2
EMNLP 2025

We introduce MAIA (Multimodal AI Assessment), a native-Italian benchmark designed for fine-grained investigation of the reasoning abilities of visual language models on videos. MAIA differs from other available video benchmarks for its design, its reasoning categories, the metric it uses, and the la

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks

Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt +14
ACL 2025short

There is an increasing trend towards evaluating NLP models with LLMs instead of human judgments, raising questions about the validity of these evaluations, as well as their reproducibility in the case of proprietary models. We provide JUDGE-BENCH, an extensible collection of 20 NLP datasets with hum…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Playpen: An Environment for Exploring Learning From Dialogue Game Feedback

Nicola Horst, Davide Mazzaccara, Antonia Schmidt, Michael Sullivan, Filippo Moment{\`e}, Luca Franceschetti +10
EMNLP 2025

Interaction between learner and feedback-giver has come into focus recently for post-training of Large Language Models (LLMs), through the use of reward models that judge the appropriateness of a model’s response. In this paper, we investigate whether Dialogue Games—goal-directed and rule-governed a

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

The Validation Gap: A Mechanistic Analysis of How Language Models Compute Arithmetic but Fail to Validate It

Leonardo Bertolazzi, Philipp Mondorf, Barbara Plank, Raffaella Bernardi
EMNLP 2025

The ability of large language models (LLMs) to validate their output and identify potential errors is crucial for ensuring robustness and reliability. However, current research indicates that LLMs struggle with self-correction, encountering significant challenges in detecting errors. While studies h

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Triangulating LLM Progress through Benchmarks, Games, and Cognitive Tests

Filippo Moment{\`e}, Alessandro Suglia, Mario Giulianelli, Ambra Ferrari, Alexander Koller, Oliver Lemon +3
EMNLP 2025

We examine three evaluation paradigms: standard benchmarks (e.g., MMLU and BBH), interactive games (e.g., Signalling Games or Taboo), and cognitive tests (e.g., for working memory or theory of mind). First, we investigate which of the former two—benchmarks or games—is most effective at discriminatin

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2024

A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences

Leonardo Bertolazzi, Albert Gatt, Raffaella Bernardi
EMNLP 2024main

The reasoning abilities of Large Language Models (LLMs) are becoming a central focus of study in NLP. In this paper, we consider the case of syllogistic reasoning, an area of deductive reasoning studied extensively in logic and cognitive psychology. Previous research has shown that pre-trained LLMs…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

Learning to Ask Informative Questions: Enhancing LLMs with Preference Optimization and Expected Information Gain

Davide Mazzaccara, Alberto Testoni, Raffaella Bernardi
EMNLP 2024finding

Questions are essential tools for acquiring the necessary information to complete information-seeking tasks. However, large language models (LLMs), especially open-source models, often perform poorly in generating informative questions, as measured by expected information gain (EIG). In this paper,…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2022

ACT-Thor: A Controlled Benchmark for Embodied Action Understanding in Simulated Environments

Michael Hanna, Federico Pedeni, Alessandro Suglia, Alberto Testoni, Raffaella Bernardi
COLING 2022main

Artificial agents are nowadays challenged to perform embodied AI tasks. To succeed, agents must understand the meaning of verbs and how their corresponding actions transform the surrounding world. In this work, we propose ACT-Thor, a novel controlled benchmark for embodied action understanding. We u…

Cited by 3Source ↗PDF ↗code ↗Scholar ↗
2021

Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy

Alberto Testoni, Raffaella Bernardi
EMNLP 2021main

Generating goal-oriented questions in Visual Dialogue tasks is a challenging and longstanding problem. State-Of-The-Art systems are shown to generate questions that, although grammatically correct, often lack an effective strategy and sound unnatural to humans. Inspired by the cognitive literature o…

Cited by 14Source ↗PDF ↗code ↗Scholar ↗