AIConfPaper
← Search

Prateek Yadav

15 accepted papers

2025

Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code

Taishi Nakamura, Mayank Mishra, Simone Tedeschi, Yekun Chai, Jason T. Stillerman, Felix Friedrich +35
COLING 2025industry

Pretrained language models are integral part of AI applications, but their high computational cost for training limits accessibility. Initiatives such as Bloom and StarCoder aim to democratize access to pretrained models for collaborative community development. Despite these efforts, such models enc…

Cited by 2Source ↗PDF ↗Scholar ↗
2025

BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions

Terry Yue Zhuo, Vu Minh Chien, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari +27
ICLR 2025oral

Task automation has been greatly empowered by the recent advances in Large Language Models (LLMs) via Python code, where the tasks range from software engineering development to general-purpose reasoning. While current benchmarks have shown that LLMs can solve tasks using programs like human develop…

Cited by 609Source ↗PDF ↗Scholar ↗
2025

Glider: Global and Local Instruction-Driven Expert Router

Pingzhi Li, Prateek Yadav, Jaehong Yoon, Jie Peng, Yi-Lin Sung, Mohit Bansal +1
EMNLP 2025

The development of performant pre-trained models has driven the advancement of routing-based expert models tailored to specific tasks. However, these methods often favor generalization over performance on held-in tasks. This limitation adversely impacts practical applicability, as real-world deploym

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2024

$\mathbb{D}^2$ Pruning: Message Passing for Balancing Diversity & Difficulty in Data Pruning

Adyasha Maharana, Prateek Yadav, Mohit Bansal
ICLR 2024poster

In recent years, data quality has emerged as an important factor for training massive models. Analytical theories suggest that higher-quality data can lead to lower test errors in models trained on a fixed data budget. Moreover, a model can be trained on a lower compute budget without compromising p…

Cited by 3Source ↗PDF ↗Scholar ↗
2024

Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy

Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal +1
ICLR 2024spotlight

Sparsely activated Mixture-of-Experts (SMoE) has shown promise to scale up the learning capacity of neural networks, however, they have issues like: ($a$) $\textit{High Memory Usage,}$ due to duplication of the network layers into multiple copies as experts; and ($b$) $\textit{Redundancy in Experts,…

Cited by 36Source ↗PDF ↗code ↗Scholar ↗
2023

Exploring Continual Learning for Code Generation Models

Prateek Yadav, Qing Sun, Hantian Ding, Xiaopeng Li, Dejiao Zhang, Ming Tan +6
ACL 2023short

Large-scale code generation models such as Copilot and CodeT5 have achieved impressive performance. However, libraries are upgraded or deprecated very frequently and re-training large-scale language models is computationally expensive. Therefore, Continual Learning (CL) is an important aspect that r…

Cited by 32Source ↗PDF ↗code ↗Scholar ↗
2023

Self-Chained Image-Language Model for Video Localization and Question Answering

Shoubin Yu, Jaemin Cho, Prateek Yadav, Mohit Bansal
NeurIPS 2023poster

Recent studies have shown promising results on utilizing large pre-trained image-language models for video question answering. While these image-language models can efficiently bootstrap the representation learning of video-language models, they typically concatenate uniformly sampled video frames a…

Cited by 183Source ↗PDF ↗code ↗Scholar ↗
2023

TIES-Merging: Resolving Interference When Merging Models

Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, Mohit Bansal
NeurIPS 2023poster

Transfer learning – i.e., further fine-tuning a pre-trained model on a downstream task – can confer significant advantages, including improved downstream performance, faster convergence, and better sample efficiency. These advantages have led to a proliferation of task-specific fine-tuned models, wh…

Cited by 290Source ↗PDF ↗code ↗Scholar ↗
2022

Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning

Swarnadeep Saha, Prateek Yadav, Mohit Bansal
ACL 2022long

Pre-trained sequence-to-sequence language models have led to widespread success in many natural language generation tasks. However, there has been relatively less work on analyzing their ability to generate structured outputs such as graphs. Unlike natural language, graphs have distinct structural a…

Cited by 10Source ↗PDF ↗code ↗Scholar ↗
2021

ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning

Swarnadeep Saha, Prateek Yadav, Lisa Bauer, Mohit Bansal
EMNLP 2021main

Recent commonsense-reasoning tasks are typically discriminative in nature, where a model answers a multiple-choice question for a certain context. Discriminative tasks are limiting because they fail to adequately evaluate the model’s ability to reason and explain predictions with underlying commonse…

Cited by 60Source ↗PDF ↗code ↗Scholar ↗
2021

multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning

Swarnadeep Saha, Prateek Yadav, Mohit Bansal
NAACL 2021long

We focus on a type of linguistic formal reasoning where the goal is to reason over explicit knowledge in the form of natural language facts and rules (Clark et al., 2020). A recent work, named PRover (Saha et al., 2020), performs such reasoning by answering a question and also generating a proof gra…

Cited by 29Source ↗PDF ↗code ↗Scholar ↗
2019

Confidence-based Graph Convolutional Networks for Semi-Supervised Learning

Shikhar Vashishth, Prateek Yadav, Manik Bhandari, Partha Talukdar
AISTATS 2019poster

Predicting properties of nodes in a graph is an important problem with applications in a variety of domains. Graph-based Semi Supervised Learning (SSL) methods aim to address this problem by labeling a small subset of the nodes as seeds, and then utilizing the graph structure to predict label scores…

Cited by 54Source ↗PDF ↗code ↗Scholar ↗
2019

HyperGCN: A New Method For Training Graph Convolutional Networks on Hypergraphs

Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, Partha Talukdar
NeurIPS 2019poster

In many real-world network datasets such as co-authorship, co-citation, email communication, etc., relationships are complex and go beyond pairwise. Hypergraphs provide a flexible and natural modeling tool to model such complex relationships. The obvious existence of such complex relationships in ma…

Cited by 607Source ↗PDF ↗code ↗Scholar ↗
2019

Lovasz Convolutional Networks

Prateek Yadav, Madhav Nimishakavi, Naganand Yadati, Shikhar Vashishth, Arun Rajkumar, Partha Talukdar
AISTATS 2019poster

Semi-supervised learning on graph structured data has received significant attention with the recent introduction of Graph Convolution Networks (GCN). While traditional methods have focused on optimizing a loss augmented with Laplacian regularization framework, GCNs perform an implicit Laplacian typ…

Cited by 17Source ↗PDF ↗code ↗Scholar ↗