AIConfPaper
← Search

Shivam Garg

12 accepted papers

2026

Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning

Vaishnavi Shrivastava, Ahmed Hassan Awadallah, Vidhisha Balachandran, Shivam Garg, Harkirat Behl, Dimitris Papailiopoulos
ICLR 2026poster

Large language models trained with reinforcement learning on verifiable rewards often inflate response length—trading brevity for accuracy. While longer reasoning can help on hard problems, many extra tokens are filler: verbose text making little progress. We introduce GFPO (Group Filtered Policy Op…

Cited by 0Source ↗code ↗Scholar ↗
2026

Wait, Wait, Wait... Why Do Reasoning Models Loop?

Charilaos Pipis, Shivam Garg, Vasilis Kontonis, Vaishnavi Shrivastava, Akshay Krishnamurthy, Dimitris Papailiopoulos
ICML 2026spotlight

Reasoning models (e.g., DeepSeek-R1) generate long chains of thought to solve harder problems, but they often loop, repeating the same text at low temperatures or with greedy decoding. We study why this happens and what role temperature plays. With open reasoning models, we find that looping is comm…

Cited by 0Source ↗Scholar ↗
2025

Discovering Data Structures: Nearest Neighbor Search and Beyond

Omar Salemohamed, Laurent Charlin, Shivam Garg, Vatsal Sharan, Gregory Valiant
NeurIPS 2025poster

We explore if it is possible to learn data structures end-to-end with neural networks, with a focus on the problem of nearest-neighbor (NN) search. We introduce a framework for data structure discovery, which adapts to the underlying data distribution and provides fine-grained control over query and…

Cited by 0Source ↗Scholar ↗
2025

Machine Unlearning via Simulated Oracle Matching

Kristian Georgiev, Roy Rinberg, Sung Min Park, Shivam Garg, Andrew Ilyas, Aleksander Madry +1
ICLR 2025poster

Machine unlearning---efficiently removing the effect of a small "forget set" of training data on a pre-trained machine learning model---has recently attracted significant research interest. Despite this interest, however, recent work shows that existing machine unlearning techniques do not hold up t…

Cited by 0Source ↗PDF ↗Scholar ↗
2022

A general class of surrogate functions for stable and efficient reinforcement learning

Sharan Vaswani, Olivier Bachem, Simone Totaro, Robert Müller, Shivam Garg, Matthieu Geist +3
AISTATS 2022poster

Common policy gradient methods rely on the maximization of a sequence of surrogate functions. In recent years, many such surrogate functions have been proposed, most without strong theoretical guarantees, leading to algorithms such as TRPO, PPO, or MPO. Rather than design yet another surrogate funct…

Cited by 19Source ↗PDF ↗code ↗Scholar ↗
2022

An Alternate Policy Gradient Estimator for Softmax Policies

Shivam Garg, Samuele Tosatto, Yangchen Pan, Martha White, Rupam Mahmood
AISTATS 2022poster

Policy gradient (PG) estimators are ineffective in dealing with softmax policies that are sub-optimally saturated, which refers to the situation when the policy concentrates its probability mass on sub-optimal actions. Sub-optimal policy saturation may arise from bad policy initialization or sudden…

Cited by 7Source ↗PDF ↗code ↗Scholar ↗
2022

How and When Random Feedback Works: A Case Study of Low-Rank Matrix Factorization

Shivam Garg, Santosh Vempala
AISTATS 2022poster

The success of gradient descent in ML and especially for learning neural networks is remarkable and robust. In the context of how the brain learns, one aspect of gradient descent that appears biologically difficult to realize (if not implausible) is that its updates rely on feedback from later layer…

Cited by 5Source ↗PDF ↗Scholar ↗
2022

What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

Shivam Garg, Dimitris Tsipras, Percy Liang, Gregory Valiant
NeurIPS 2022accept

In-context learning is the ability of a model to condition on a prompt sequence consisting of in-context examples (input-output pairs corresponding to some task) along with a new query input, and generate the corresponding output. Crucially, in-context learning happens only at inference time without…

Cited by 491Source ↗PDF ↗code ↗Scholar ↗
2021

Reward Identification in Inverse Reinforcement Learning

Kuno Kim, Shivam Garg, Kirankumar Shiragur, Stefano Ermon
ICML 2021spotlight

We study the problem of reward identifiability in the context of Inverse Reinforcement Learning (IRL). The reward identifiability question is critical to answer when reasoning about the effectiveness of using Markov Decision Processes (MDPs) as computational models of real world decision makers in o…

Cited by 51Source ↗PDF ↗Scholar ↗
2020

Gradient Temporal-Difference Learning with Regularized Corrections

Sina Ghiassian, Andrew Patterson, Shivam Garg, Dhawal Gupta, Adam White, Martha White
ICML 2020poster

It is still common to use Q-learning and temporal difference (TD) learning{—}even though they have divergence issues and sound Gradient TD alternatives exist{—}because divergence seems rare and they typically perform well. However, recent work with large neural network learning systems reveals that…

Cited by 59Source ↗PDF ↗code ↗Scholar ↗
2020

Sample Amplification: Increasing Dataset Size even when Learning is Impossible

Brian Axelrod, Shivam Garg, Vatsal Sharan, Gregory Valiant
ICML 2020poster

Given data drawn from an unknown distribution, D, to what extent is it possible to “amplify” this dataset and faithfully output an even larger set of samples that appear to have been drawn from D? We formalize this question as follows: an (n,m) amplification procedure takes as input n independent dr…

Cited by 17Source ↗PDF ↗Scholar ↗