AIConfPaper
← Search

Oliver Lemon

10 accepted papers

2025

Playpen: An Environment for Exploring Learning From Dialogue Game Feedback

Nicola Horst, Davide Mazzaccara, Antonia Schmidt, Michael Sullivan, Filippo Moment{\`e}, Luca Franceschetti +10
EMNLP 2025

Interaction between learner and feedback-giver has come into focus recently for post-training of Large Language Models (LLMs), through the use of reward models that judge the appropriateness of a model’s response. In this paper, we investigate whether Dialogue Games—goal-directed and rule-governed a

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Triangulating LLM Progress through Benchmarks, Games, and Cognitive Tests

Filippo Moment{\`e}, Alessandro Suglia, Mario Giulianelli, Ambra Ferrari, Alexander Koller, Oliver Lemon +3
EMNLP 2025

We examine three evaluation paradigms: standard benchmarks (e.g., MMLU and BBH), interactive games (e.g., Signalling Games or Taboo), and cognitive tests (e.g., for working memory or theory of mind). First, we investigate which of the former two—benchmarks or games—is most effective at discriminatin

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2024

AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding

Alessandro Suglia, Claudio Greco, Katie Baker, Jose L. Part, Ioannis Papaioannou, Arash Eshghi +2
EMNLP 2024finding

AI personal assistants deployed via robots or wearables require embodied understanding to collaborate with humans effectively. However, current Vision-Language Models (VLMs) primarily focus on third-person view videos, neglecting the richness of egocentric perceptual experience. To address this gap,…

Cited by 9Source ↗PDF ↗code ↗Scholar ↗
2024

Clarifying Completions: Evaluating How LLMs Respond to Incomplete Questions

Angus Addlesee, Oliver Lemon, Arash Eshghi
COLING 2024main

People understand and produce language incrementally on a word by word basis. This gives rise to many characteristic conversational phenomena including long mid-sentence pauses that are followed by incremental clarification requests (iCRs) intended to recover the rest of the truncated turn (see Fig.…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

Lost in Space: Probing Fine-grained Spatial Understanding in Vision and Language Resamplers

Georgios Pantazopoulos, Alessandro Suglia, Oliver Lemon, Arash Eshghi
NAACL 2024short

An effective method for combining frozen large language models (LLM) and visual encoders involves a resampler module that creates a ‘visual prompt’ which is provided to the LLM, along with the textual prompt. While this approach has enabled impressive performance across many coarse-grained tasks lik…

Cited by 3Source ↗PDF ↗code ↗Scholar ↗
2024

RECANTFormer: Referring Expression Comprehension with Varying Numbers of Targets

Bhathiya Hemanthage, Hakan Bilen, Phil Bartie, Christian Dondrup, Oliver Lemon
EMNLP 2024main

The Generalized Referring Expression Comprehension (GREC) task extends classic REC by generating image bounding boxes for objects referred to in natural language expressions, which may indicate zero, one, or multiple targets. This generalization enhances the practicality of REC models for diverse re…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

Shaking Up VLMs: Comparing Transformers and Structured State Space Models for Vision & Language Modeling

Georgios Pantazopoulos, Malvina Nikandrou, Alessandro Suglia, Oliver Lemon, Arash Eshghi
EMNLP 2024main

This study explores replacing Transformers in Visual Language Models (VLMs) with Mamba, a recent structured state space model (SSM) that demonstrates promising performance in sequence modeling. We test models up to 3B parameters under controlled conditions, showing that Mamba-based VLMs outperforms…

Cited by 2Source ↗PDF ↗code ↗Scholar ↗
2023

Multitask Multimodal Prompted Training for Interactive Embodied Task Completion

Georgios Pantazopoulos, Malvina Nikandrou, Amit Parekh, Bhathiya Hemanthage, Arash Eshghi, Ioannis Konstas +3
EMNLP 2023long main

Interactive and embodied tasks pose at least two fundamental challenges to existing Vision \& Language (VL) models, including 1) grounding language in trajectories of actions and observations, and 2) referential disambiguation. To tackle these challenges, we propose an Embodied MultiModal Agent (EMM…

Cited by 0Source ↗Scholar ↗
2020

Imagining Grounded Conceptual Representations from Perceptual Information in Situated Guessing Games

Alessandro Suglia, Antonio Vergari, Ioannis Konstas, Yonatan Bisk, Emanuele Bastianelli, Andrea Vanzo +1
COLING 2020main

In visual guessing games, a Guesser has to identify a target object in a scene by asking questions to an Oracle. An effective strategy for the players is to learn conceptual representations of objects that are both discriminative and expressive enough to ask questions and guess correctly. However, a…

Cited by 10Source ↗PDF ↗code ↗Scholar ↗