← Search

Frank Guerin

13 accepted papers

2024

An Open-Source Data Contamination Report for Large Language Models

EMNLP 2024finding

Data contamination in model evaluation has become increasingly prevalent with the growing popularity of large language models. It allows models to “cheat” via memorisation instead of displaying true capabilities. Therefore, contamination analysis has become an crucial part of reliable model evaluati…

2024

GPTEval: A Survey on Assessments of ChatGPT and GPT-4

COLING 2024main

The emergence of ChatGPT has generated much speculation in the press about its potential to disrupt social and economic systems. Its astonishing language ability has aroused strong curiosity among scholars about its performance in different domains. There have been many studies evaluating the abilit…

Cited by 125SourcePDFScholar
2024

LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Construction

AAAI 2024technical

Data contamination in evaluation is getting increasingly prevalent with the emergence of language models pre-trained on super large, automatically crawled corpora. This problem leads to significant challenges in the accurate assessment of model capabilities and generalisations. In this paper, we pro…

2023

Compressing Context to Enhance Inference Efficiency of Large Language Models

EMNLP 2023long main

Large language models (LLMs) achieved remarkable performance across various tasks. However, they face challenges in managing long documents and extended conversations, due to significantly increased computational requirements, both in memory and inference time, and potential context truncation when…

Cited by 0SourcecodeScholar
2023

Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregation

ACL 2023long

Incorporating external graph knowledge into neural chatbot models has been proven effective for enhancing dialogue generation. However, in conventional graph neural networks (GNNs), message passing on a graph is independent from text, resulting in the graph representation hidden space differing from…

2023

Metaphor Detection via Explicit Basic Meanings Modelling

ACL 2023short

One noticeable trend in metaphor detection is the embrace of linguistic theories such as the metaphor identification procedure (MIP) for model architecture design. While MIP clearly defines that the metaphoricity of a lexical unit is determined based on the contrast between its contextual meaning an…

2022

CM-Gen: A Neural Framework for Chinese Metaphor Generation with Explicit Context Modelling

COLING 2022main

Nominal metaphors are frequently used in human language and have been shown to be effective in persuading, expressing emotion, and stimulating interest. This paper tackles the problem of Chinese Nominal Metaphor (NM) generation. We introduce a novel multitask framework, which jointly optimizes three…

2022

EtriCA: Event-Triggered Context-Aware Story Generation Augmented by Cross Attention

EMNLP 2022finding

One of the key challenges of automatic story generation is how to generate a long narrative that can maintain fluency, relevance, and coherence. Despite recent progress, current story generation systems still face the challenge of how to effectively capture contextual and event features, which has a…

2020

Latent Space Factorisation and Manipulation via Matrix Subspace Projection

ICML 2020poster

We tackle the problem disentangling the latent space of an autoencoder in order to separate labelled attribute information from other characteristic information. This then allows us to change selected attributes while preserving other information. Our method, matrix subspace projection, is much simp…

2019

Adapting Everyday Manipulation Skills to Varied Scenarios

ICRA 2019poster

We address the problem of executing tool-using manipulation skills in scenarios where the objects to be used may vary. We assume that point clouds of the tool and target object can be obtained, but no interpretation or further knowledge about these objects is provided. The system must interpret the…

Cited by 32SourceScholar