← Search

Denis Shevelev

3 accepted papers

2024

MERA: A Comprehensive LLM Evaluation in Russian

ACL 2024long

Over the past few years, one of the most notable advancements in AI research has been in foundation models (FMs), headlined by the rise of language models (LMs). However, despite researchers’ attention and the rapid growth in LM application, the capabilities, limitations, and associated risks still…

2022

TAPE: Assessing Few-shot Russian Language Understanding

EMNLP 2022finding

Recent advances in zero-shot and few-shot learning have shown promise for a scope of research and practical purposes. However, this fast-growing area lacks standardized evaluation suites for non-English languages, hindering progress outside the Anglo-centric paradigm. To address this line of researc…

2020

Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian

COLING 2020main

The paper introduces two Russian machine reading comprehension (MRC) datasets, called MuSeRC and RuCoS, which require reasoning over multiple sentences and commonsense knowledge to infer the answer. The former follows the design of MultiRC, while the latter is a counterpart of the ReCoRD dataset. Th…