2026
Post-Training with Policy Gradients: Optimality and the Base Model Barrier
ICML 2026spotlight
We study post-training linear autoregressive models with outcome and process rewards. Given a context $\boldsymbol{x}$, the model must predict the response $\boldsymbol{y} \in \mathcal{Y}^N$, a sequence of length $N$ that satisfies a standard $\gamma$ margin assumption extended to sequences. We prov…