AIConfPaper
← Search

David Stap

8 accepted papers

2026

xLSTM Distillation: Achieving Teacher-Student Parity Through Efficient Hybrid Architectures

Lukas Hauzenberger, Niklas Schmidinger, Thomas Schmied, Anamaria-Roberta Hartl, David Stap, Pieter-Jan Hoedt +3
ICML 2026poster

There have been numerous attempts to distill quadratic attention-based LLMs into sub-quadratic linearized architectures. However, despite extensive research, such distilled models often fail to match the performance of their teacher LLMs on various downstream tasks. We set out the goal of lossless d…

Cited by 0Source ↗Scholar ↗
2025

Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?

Seth Aycock, David Stap, Di Wu, Christof Monz, Khalil Sima'an
ICLR 2025spotlight

Extremely low-resource (XLR) languages lack substantial corpora for training NLP models, motivating the use of all available resources such as dictionaries and grammar books. Machine Translation from One Book (Tanzer et al., 2024) suggests that prompting long-context LLMs with one grammar book enabl…

Cited by 5Source ↗PDF ↗Scholar ↗
2025

MMTEB: Massive Multilingual Text Embedding Benchmark

Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap +76
ICLR 2025poster

Text embeddings are typically evaluated on a narrow set of tasks, limited in terms of languages, domains, and task types. To circumvent this limitation and to provide a more comprehensive evaluation, we introduce the Massive Multilingual Text Embedding Benchmark (MMTEB) -- a large-scale community-dr…

Cited by 3Source ↗PDF ↗code ↗Scholar ↗
2024

How Far can 100 Samples Go? Unlocking Zero-Shot Translation with Tiny Multi-Parallel Data

Di Wu, Shaomu Tan, Yan Meng, David Stap, Christof Monz
ACL 2024findings

Zero-shot translation aims to translate between language pairs not seen during training in Multilingual Machine Translation (MMT) and is widely considered an open problem. A common, albeit resource-consuming, solution is to add as many related translation directions as possible to the training corpu…

Cited by 2Source ↗PDF ↗code ↗Scholar ↗
2024

The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities

David Stap, Eva Hasler, Bill Byrne, Christof Monz, Ke Tran
ACL 2024long

Fine-tuning large language models (LLMs) for machine translation has shown improvements in overall translation quality. However, it is unclear what is the impact of fine-tuning on desirable LLM behaviors that are not present in neural machine translation models, such as steerability, inherent docume…

Cited by 6Source ↗PDF ↗code ↗Scholar ↗
2023

Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens

David Stap, Vlad Niculae, Christof Monz
EMNLP 2023long findings

We argue that translation quality alone is not a sufficient metric for measuring knowledge transfer in multilingual neural machine translation. To support this claim, we introduce Representational Transfer Potential (RTP), which measures representational similarities between languages. We show that…

Cited by 0Source ↗Scholar ↗
2022

Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks

Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik +29
EMNLP 2022main

How well can NLP models generalize to a variety of unseen tasks when provided with task instructions? To address this question, we first introduce Super-NaturalInstructions, a benchmark of 1,616 diverse NLP tasks and their expert-written instructions. Our collection covers 76 distinct task types, in…

Cited by 582Source ↗PDF ↗code ↗Scholar ↗