Nikhil Anand — accepted AI-conference papers

2025

Mixture of Parrots: Experts improve memorization more than reasoning

Samy Jelassi, Clara Mohri, David Brandfonbrener, Alex Gu, Nikhil Vyas, Nikhil Anand +4

ICLR 2025poster

The Mixture-of-Experts (MoE) architecture enables a significant increase in the total number of model parameters with minimal computational overhead. However, it is not clear what performance tradeoffs, if any, exist between MoEs and standard dense transformers. In this paper, we show that as we i…

Cited by 3 Source ↗PDF ↗Scholar ↗

2023

Influence Scores at Scale for Efficient Language Data Sampling

Nikhil Anand, Joshua Tan, Maria Minakova

EMNLP 2023long main

Modern ML systems ingest data aggregated from diverse sources, such as synthetic, human-annotated, and live customer traffic. Understanding \textit{which} examples are important to the performance of a learning algorithm is crucial for efficient model training. Recently, a growing body of literature…

Cited by 0 Source ↗Scholar ↗