2024
On Leakage of Code Generation Evaluation Datasets
EMNLP 2024finding
In this paper, we consider contamination by code generation test sets, in particular in their use in modern large language models.We discuss three possible sources of such contamination and show findings supporting each of them: (i) direct data leakage, (ii) indirect data leakage through the use of…