AIConfPaper
← Search

Sherzod Hakimov

9 accepted papers

2025

Playpen: An Environment for Exploring Learning From Dialogue Game Feedback

Nicola Horst, Davide Mazzaccara, Antonia Schmidt, Michael Sullivan, Filippo Moment{\`e}, Luca Franceschetti +10
EMNLP 2025

Interaction between learner and feedback-giver has come into focus recently for post-training of Large Language Models (LLMs), through the use of reward models that judge the appropriateness of a model’s response. In this paper, we investigate whether Dialogue Games—goal-directed and rule-governed a

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models

Sherzod Hakimov, Yerkezhan Abdullayeva, Kushal Koshti, Antonia Schmidt, Yan Weiser, Anne Beyer +1
COLING 2025main

While the situation has improved for text-only models, it again seems to be the case currently that multimodal (text and image) models develop faster than ways to evaluate them. In this paper, we bring a recently developed evaluation paradigm from text models to multimodal models, namely evaluation…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

M2SA: Multimodal and Multilingual Model for Sentiment Analysis of Tweets

Gaurish Thakkar, Sherzod Hakimov, Marko Tadić
COLING 2024main

In recent years, multimodal natural language processing, aimed at learning from diverse data types, has garnered significant attention. However, there needs to be more clarity when it comes to analysing multimodal tasks in multi-lingual contexts. While prior studies on sentiment analysis of tweets h…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2024

Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft

Chalamalasetti Kranti, Sherzod Hakimov, David Schlangen
EMNLP 2024finding

In the Minecraft Collaborative Building Task, two players collaborate: an Architect (A) provides instructions to a Builder (B) to assemble a specified structure using 3D blocks. In this work, we investigate the use of large language models (LLMs) to predict the sequence of actions taken by the Build…

Cited by 2Source ↗PDF ↗Scholar ↗
2024

Sharing the Cost of Success: A Game for Evaluating and Learning Collaborative Multi-Agent Instruction Giving and Following Policies

Philipp Sadler, Sherzod Hakimov, David Schlangen
COLING 2024main

In collaborative goal-oriented settings, the participants are not only interested in achieving a successful outcome, but do also implicitly negotiate the effort they put into the interaction (by adapting to each other). In this work, we propose a challenging interactive reference game that requires…

Cited by 4Source ↗PDF ↗code ↗Scholar ↗
2023

Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks

Sherzod Hakimov, David Schlangen
ACL 2023findings

Large language models have demonstrated robust performance on various language tasks using zero-shot or few-shot learning paradigms. While being actively researched, multimodal models that can additionally handle images as input have yet to catch up in size and generality with language-only models.…

Cited by 6Source ↗PDF ↗code ↗Scholar ↗
2023

Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers

Philipp Sadler, Sherzod Hakimov, David Schlangen
ACL 2023findings

The ability to pick up on language signals in an ongoing interaction is crucial for future machine learning models to collaborate and interact with humans naturally. In this paper, we present an initial study that evaluates intra-episodic feedback given in a collaborative setting. We use a referenti…

Cited by 2Source ↗PDF ↗code ↗Scholar ↗
2023

clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents

Kranti CH, Jana Götze, Sherzod Hakimov, Brielen Madureira, Philipp Sadler, David Schlangen
EMNLP 2023long main

Recent work has proposed a methodology for the systematic evaluation of "Situated Language Understanding Agents" --- agents that operate in rich linguistic and non-linguistic contexts --- through testing them in carefully constructed interactive settings. Other recent work has argued that Large Lang…

Cited by 62Source ↗code ↗Scholar ↗
2022

MM-Claims: A Dataset for Multimodal Claim Detection in Social Media

Gullal Singh Cheema, Sherzod Hakimov, Abdul Sittar, Eric Müller-Budack, Christian Otto, Ralph Ewerth
NAACL 2022findings

In recent years, the problem of misinformation on the web has become widespread across languages, countries, and various social media platforms. Although there has been much work on automated fake news detection, the role of images and their variety are not well explored. In this paper, we investiga…

Cited by 23Source ↗PDF ↗code ↗Scholar ↗