← Search

Frederic Bechet

4 accepted papers

2025

Factual Knowledge Assessment of Language Models Using Distractors

COLING 2025main

Language models encode extensive factual knowledge within their parameters. The accurate assessment of this knowledge is crucial for understanding and improving these models. In the literature, factual knowledge assessment often relies on cloze sentences, which can lead to erroneous conclusions due…

2025

Part-Of-Speech Sensitivity of Routers in Mixture of Experts Models

COLING 2025main

This study investigates the behavior of model-integrated routers in Mixture of Experts (MoE) models, focusing on how tokens are routed based on their linguistic features, specifically Part-of-Speech (POS) tags. The goal is to explore across different MoE architectures whether experts specialize in p…

Cited by 1SourcePDFScholar
2025

Statistical Deficiency for Task Inclusion Estimation

ACL 2025long

Tasks are central in machine learning, as they are the most natural objects to assess the capabilities of current models. The trend is to build general models able to address any task. Even though transfer learning and multitask learning try to leverage the underlying task space, no well-founded too…

Cited by 0SourcePDFScholar
2024

A linguistically-motivated evaluation methodology for unraveling model’s abilities in reading comprehension tasks

EMNLP 2024main

We introduce an evaluation methodology for reading comprehension tasks based on the intuition that certain examples, by the virtue of their linguistic complexity, consistently yield lower scores regardless of model size or architecture. We capitalize on semantic frame annotation for characterizing t…