AIConfPaper
← Search

Mike Zhang

13 accepted papers

2026

Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation

Israfel Salazar, Manuel Fernández Burda, Shayekh Bin Islam, Arshia Soltani Moakhar, Shivalika Singh, Fabian Farestam +39
ICLR 2026poster

The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While multilingual benchmarks have expanded, both in size and language, many rely on translations of English datasets, failing t…

Cited by 0Source ↗code ↗Scholar ↗
2025

All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages

Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana, Noor Ahsan, Nevasini Sasikumar, Omkar Thawakar +63
CVPR 2025highlight

Existing Large Multimodal Models (LMMs) generally focus on only a few regions and languages. As LMMs continue to improve, it is increasingly important to ensure they understand cultural contexts, respect local sensitivities, and support low-resource languages, all while effectively integrating corr…

Cited by 14Source ↗PDF ↗code ↗Scholar ↗
2025

INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge

Angelika Romanou, Negar Foroutan, Anna Sotnikova, Sree Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary +51
ICLR 2025spotlight

The performance differential of large language models (LLM) between languages hinders their effective deployment in many regions, inhibiting the potential economic and societal value of generative AI tools in many communities. However, the development of functional LLMs in many languages (i.e., mult…

Cited by 9Source ↗PDF ↗Scholar ↗
2025

SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models

Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle +48
NAACL 2025long

Large Language Models (LLMs) reproduce and exacerbate the social biases present in their training data, and resources to quantify this issue are limited. While research has attempted to identify and mitigate such biases, most efforts have been concentrated around English, lagging the rapid advanceme…

Cited by 1Source ↗PDF ↗Scholar ↗
2024

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

Shivalika Singh, Freddie Vargus, Daniel D’souza, Börje Karlsson, Abinaya Mahendiran, Wei-Yin Ko +27
ACL 2024long

Datasets are foundational to many breakthroughs in modern artificial intelligence. Many recent achievements in the space of natural language processing (NLP) can be attributed to the fine-tuning of pre-trained models on a diverse set of tasks that enables a large language model (LLM) to respond to i…

Cited by 94Source ↗PDF ↗Scholar ↗
2024

Can Humans Identify Domains?

Maria Barrett, Max Müller-Eberstein, Elisa Bassignana, Amalie Brogaard Pauli, Mike Zhang, Rob van der Goot
COLING 2024main

Textual domain is a crucial property within the Natural Language Processing (NLP) community due to its effects on downstream model performance. The concept itself is, however, loosely defined and, in practice, refers to any non-typological property, such as genre, topic, medium or style of a documen…

Cited by 2Source ↗PDF ↗code ↗Scholar ↗
2024

Tag Map: A Text-Based Map for Spatial Reasoning and Navigation with Large Language Models

Mike Zhang, Kaixian Qu, Vaishakh Patil, Cesar Cadena, Marco Hutter
CoRL 2024poster

Large Language Models (LLM) have emerged as a tool for robots to generate task plans using common sense reasoning. For the LLM to generate actionable plans, scene context must be provided, often through a map. Recent works have shifted from explicit maps with fixed semantic classes to implicit open…

Cited by 3Source ↗Scholar ↗
2023

ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain

Mike Zhang, Rob van der Goot, Barbara Plank
ACL 2023long

The increasing number of benchmarks for Natural Language Processing (NLP) tasks in the computational job market domain highlights the demand for methods that can handle job-related tasks such as skill extraction, skill classification, job title classification, and de-identification. While some appro…

Cited by 19Source ↗PDF ↗code ↗Scholar ↗
2022

Evidence > Intuition: Transferability Estimation for Encoder Selection

Elisa Bassignana, Max Müller-Eberstein, Mike Zhang, Barbara Plank
EMNLP 2022main

With the increase in availability of large pre-trained language models (LMs) in Natural Language Processing (NLP), it becomes critical to assess their fit for a specific target task a priori—as fine-tuning the entire space of available LMs is computationally prohibitive and unsustainable. However, e…

Cited by 16Source ↗PDF ↗code ↗Scholar ↗
2022

Experimental Standards for Deep Learning in Natural Language Processing Research

Dennis Ulmer, Elisa Bassignana, Max Müller-Eberstein, Daniel Varab, Mike Zhang, Rob van der Goot +2
EMNLP 2022finding

The field of Deep Learning (DL) has undergone explosive growth during the last decade, with a substantial impact on Natural Language Processing (NLP) as well. Yet, compared to more established disciplines, a lack of common experimental standards remains an open challenge to the field at large. Start…

Cited by 10Source ↗PDF ↗code ↗Scholar ↗
2022

SkillSpan: Hard and Soft Skill Extraction from English Job Postings

Mike Zhang, Kristian Jensen, Sif Sonniks, Barbara Plank
NAACL 2022long

Skill Extraction (SE) is an important and widely-studied task useful to gain insights into labor market dynamics. However, there is a lacuna of datasets and annotation guidelines; available datasets are few and contain crowd-sourced labels on the span-level or labels from a predefined skill inventor…

Cited by 76Source ↗PDF ↗code ↗Scholar ↗