AIConfPaper
← Search

Benjamin Piwowarski

13 accepted papers

2025

Context Copying Modulation: The Role of Entropy Neurons in Managing Parametric and Contextual Knowledge Conflicts

Zineddine Tighidet, Andrea Mogini, Hedi Ben younes, Jiali Mei, Patrick Gallinari, Benjamin Piwowarski
EMNLP 2025

The behavior of Large Language Models (LLMs) when facing contextual information that conflicts with their internal parametric knowledge is inconsistent, with no generally accepted explanation for the expected outcome distribution. Recent work has identified in autoregressive transformer models a cla

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

MEXMA: Token-level objectives improve sentence representations

João Maria Janeiro, Benjamin Piwowarski, Patrick Gallinari, Loic Barrault
ACL 2025long

Cross-lingual sentence encoders (CLSE) create fixed-size sentence representations with aligned translations. Current pre-trained CLSE approaches use sentence-level objectives only. This can lead to loss of information, especially for tokens, which then degrades the sentence representation. We propos…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Mixture of Languages: Improved Multilingual Encoders Through Language Grouping

Jo{\~a}o Maria Janeiro, Belen Alastruey, Francisco Massa, Maha Elbayad, Benjamin Piwowarski, Patrick Gallinari +1
EMNLP 2025

We propose Mixture of Languages (MoL), a new strategy to pretrain largely multilingual encoders. Recent work in this field has relied on training transformer encoders on a large amount of multilingual data, with all parameters shared across all languages, without studying how to optimally balance la

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Structural Deep Encoding for Table Question Answering

Raphaël Mouravieff, Benjamin Piwowarski, Sylvain Lamprier
ACL 2025finding

Although Transformers-based architectures excel at processing textual information, their naive adaptation for tabular data often involves flattening the table structure. This simplification can lead to the loss of essential inter-dependencies between rows, columns, and cells, while also posing scala…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2024

Learning Relational Decomposition of Queries for Question Answering from Tables

Raphaël Mouravieff, Benjamin Piwowarski, Sylvain Lamprier
ACL 2024long

Table Question-Answering involves both understanding the natural language query and grounding it in the context of the input table to extract relevant information. In this context, many methods have highlighted the benefits of intermediate pre-training using SQL queries. However, while most approach…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2022

Generative Cooperative Networks for Natural Language Generation

Sylvain Lamprier, Thomas Scialom, Antoine Chaffin, Vincent Claveau, Ewa Kijak, Jacopo Staiano +1
ICML 2022spotlight

Generative Adversarial Networks (GANs) have known a tremendous success for many continuous generation tasks, especially in the field of image generation. However, for discrete outputs such as language, optimizing GANs remains an open problem with many instabilities, as no gradient can be properly ba…

Cited by 23Source ↗PDF ↗Scholar ↗
2021

Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation

Clement Rebuffel, Thomas Scialom, Laure Soulier, Benjamin Piwowarski, Sylvain Lamprier, Jacopo Staiano +2
EMNLP 2021main

QuestEval is a reference-less metric used in text-to-text tasks, that compares the generated summaries directly to the source text, by automatically asking and answering questions. Its adaptation to Data-to-Text tasks is not straightforward, as it requires multimodal Question Generation and Answerin…

Cited by 30Source ↗PDF ↗code ↗Scholar ↗
2021

QuestEval: Summarization Asks for Fact-based Evaluation

Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, Alex Wang +1
EMNLP 2021main

Summarization evaluation remains an open research problem: current metrics such as ROUGE are known to be limited and to correlate poorly with human judgments. To alleviate this issue, recent work has proposed evaluation metrics which rely on question answering models to assess whether a summary cont…

Cited by 281Source ↗PDF ↗code ↗Scholar ↗
2021

Skim-Attention: Learning to Focus via Document Layout

Laura Nguyen, Thomas Scialom, Jacopo Staiano, Benjamin Piwowarski
EMNLP 2021finding

Transformer-based pre-training techniques of text and layout have proven effective in a number of document understanding tasks. Despite this success, multimodal pre-training models suffer from very high computational and memory costs. Motivated by human reading strategies, this paper presents Skim-A…

Cited by 10Source ↗PDF ↗code ↗Scholar ↗
2021

To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs

Thomas Scialom, Paul-Alexis Dray, Jacopo Staiano, sylvain lamprier, Benjamin Piwowarski
NeurIPS 2021poster

Due to the discrete nature of words, language GANs require to be optimized from rewards provided by discriminator networks, via reinforcement learning methods. This is a much harder setting than for continuous tasks, which enjoy gradient flows from discriminators to generators, usually leading to dr…

Cited by 23Source ↗PDF ↗Scholar ↗
2020

ColdGANs: Taming Language GANs with Cautious Sampling Strategies

Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano
NeurIPS 2020poster

Training regimes based on Maximum Likelihood Estimation (MLE) suffer from known limitations, often leading to poorly generated text sequences that lack of coherence, factualness, and are prone to repetitions. At the root of these limitations is the mismatch between training and inference, i.e. the s…

Cited by 23Source ↗PDF ↗code ↗Scholar ↗
2020

Discriminative Adversarial Search for Abstractive Summarization

Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano
ICML 2020poster

We introduce a novel approach for sequence decoding, Discriminative Adversarial Search (DAS), which has the desirable properties of alleviating the effects of exposure bias without requiring external metrics. Inspired by Generative Adversarial Networks (GANs), wherein a discriminator is used to impr…

Cited by 43Source ↗PDF ↗code ↗Scholar ↗
2019

Context-Aware Zero-Shot Learning for Object Recognition

Eloi Zablocki, Patrick Bordes, Laure Soulier, Benjamin Piwowarski, Patrick Gallinari
ICML 2019oral

Zero-Shot Learning (ZSL) aims at classifying unlabeled objects by leveraging auxiliary knowledge, such as semantic representations. A limitation of previous approaches is that only intrinsic properties of objects, e.g. their visual appearance, are taken into account while their context, e.g. the sur…

Cited by 43Source ↗PDF ↗Scholar ↗