Dheeraj Mekala — accepted AI-conference papers

2026

Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments

Romain Froger, Amine Benhalloum, Andrey Rusakov, Dheeraj Mekala, Emilien Garreau, Gerard Moreno-Torres Bertran +18

ICLR 2026oral

We introduce **Gaia2**, a benchmark for evaluating large language model agents in realistic, asynchronous environments. Unlike prior static or synchronous evaluations, Gaia2 introduces scenarios where environments evolve independently of agent actions, requiring agents to operate under temporal cons…

Cited by 0 Source ↗Scholar ↗

2024

DOCMASTER: A Unified Platform for Annotation, Training, & Inference in Document Question-Answering

Alex Nguyen, Zilong Wang, Jingbo Shang, Dheeraj Mekala

NAACL 2024system demonstrations

The application of natural language processing models to PDF documents is pivotal for various business applications yet the challenge of training models for this purpose persists in businesses due to specific hurdles. These include the complexity of working with PDF formats that necessitate parsing…

Cited by 3 Source ↗PDF ↗Scholar ↗

2024

MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimization

Yasaman Jafari, Dheeraj Mekala, Rose Yu, Taylor Berg-Kirkpatrick

EMNLP 2024finding

RL-based techniques can be employed to search for prompts that, when fed into a target language model, maximize a set of user-specified reward functions. However, in many target applications, the natural reward functions are in tension with one another – for example, content preservation vs. style m…

Cited by 5 Source ↗PDF ↗Scholar ↗

2024

Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models

Dheeraj Mekala, Alex Nguyen, Jingbo Shang

ACL 2024findings

Instruction-tuning language models has become a crucial step in aligning them for general use. Typically, this process involves extensive training on large datasets, incurring high training costs. In this paper, we introduce a novel training data selection based on the learning percentage of the sam…

Cited by 20 Source ↗PDF ↗code ↗Scholar ↗

2024

TOOLVERIFIER: Generalization to New Tools via Self-Verification

Dheeraj Mekala, Jason E Weston, Jack Lanchantin, Roberta Raileanu, Maria Lomeli, Jingbo Shang +1

EMNLP 2024finding

Teaching language models to use tools is an important milestone towards building general assistants, but remains an open problem. While there has been significant progress on learning to use specific tools via fine-tuning, language models still struggle with learning how to robustly use new tools fr…

Cited by 8 Source ↗PDF ↗code ↗Scholar ↗

2023

A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approaches

Zihan Wang, Tianle Wang, Dheeraj Mekala, Jingbo Shang

ACL 2023findings

Extremely Weakly Supervised Text Classification (XWS-TC) refers to text classification based on minimal high-level human guidance, such as a few label-indicative seed words or classification instructions. There are two mainstream approaches for XWS-TC, however, never being rigorously compared: (1) t…

Cited by 9 Source ↗PDF ↗code ↗Scholar ↗

2023

SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank

Dheeraj Mekala, Adithya Samavedhi, Chengyu Dong, Jingbo Shang

EMNLP 2023long findings

Deep neural classifiers trained with cross-entropy loss (CE loss) often suffer from poor calibration, necessitating the task of out-of-distribution (OOD) detection. Traditional supervised OOD detection methods require expensive manual annotation of in-distribution and OOD samples. To address the ann…

Cited by 0 Source ↗code ↗Scholar ↗

2023

ZEROTOP: Zero-Shot Task-Oriented Semantic Parsing using Large Language Models

Dheeraj Mekala, Jason Andrew Wolfe, Subhro Roy

EMNLP 2023short main

We explore the use of large language models (LLMs) for zero-shot semantic parsing. Semantic parsing involves mapping natural language utterances to task-specific meaning representations. LLMs are generally trained on publicly available text and code and cannot be expected to directly generalize to d…

Cited by 0 Source ↗Scholar ↗

2022

LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification

Dheeraj Mekala, Chengyu Dong, Jingbo Shang

EMNLP 2022finding

Weakly supervised text classification methods typically train a deep neural classifier based on pseudo-labels. The quality of pseudo-labels is crucial to final performance but they are inevitably noisy due to their heuristic nature, so selecting the correct ones has a huge potential for performance…

Cited by 21 Source ↗PDF ↗code ↗Scholar ↗

2022

Leveraging QA Datasets to Improve Generative Data Augmentation

Dheeraj Mekala, Tu Vu, Timo Schick, Jingbo Shang

EMNLP 2022main

The ability of generative language models (GLMs) to generate text has improved considerably in the last few years, enabling their use for generative data augmentation. In this work, we propose CONDA, an approach to further improve GLM’s ability to generate synthetic data by reformulating data genera…

Cited by 19 Source ↗PDF ↗code ↗Scholar ↗

2022

Progressive Sentiment Analysis for Code-Switched Text Data

Sudhanshu Ranjan, Dheeraj Mekala, Jingbo Shang

EMNLP 2022finding

Multilingual transformer language models have recently attracted much attention from researchers and are used in cross-lingual transfer learning for many NLP tasks such as text classification and named entity recognition.However, similar methods for transfer learning from monolingual text to code-sw…

Cited by 4 Source ↗PDF ↗code ↗Scholar ↗

2021

BFClass: A Backdoor-free Text Classification Framework

Zichao Li, Dheeraj Mekala, Chengyu Dong, Jingbo Shang

EMNLP 2021finding

Backdoor attack introduces artificial vulnerabilities into the model by poisoning a subset of the training data via injecting triggers and modifying labels. Various trigger design strategies have been explored to attack text classifiers, however, defending such attacks remains an open problem. In th…

Cited by 30 Source ↗PDF ↗Scholar ↗

2021

Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data

Dheeraj Mekala, Varun Gangal, Jingbo Shang

EMNLP 2021main

Existing text classification methods mainly focus on a fixed label set, whereas many real-world applications require extending to new fine-grained classes as the number of samples per label increases. To accommodate such requirements, we introduce a new problem called coarse-to-fine grained classifi…

Cited by 25 Source ↗PDF ↗code ↗Scholar ↗

2021

X-Class: Text Classification with Extremely Weak Supervision

Zihan Wang, Dheeraj Mekala, Jingbo Shang

NAACL 2021long

In this paper, we explore text classification with extremely weak supervision, i.e., only relying on the surface text of class names. This is a more challenging setting than the seed-driven weak supervision, which allows a few seed words per class. We opt to attack this problem from a representation…

Cited by 94 Source ↗PDF ↗code ↗Scholar ↗