Alexander Havrilla — accepted AI-conference papers

2024

GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements

Alexander Havrilla, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro +1

ICML 2024poster

State-of-the-art language models can exhibit reasoning refinement capabilities on math, science or coding tasks. However, recent work demonstrates that even the best models struggle to identify *when and where to refine* without access to external feedback. In this paper, we propose Stepwise ORMs (*…

Cited by 52 Source ↗PDF ↗Scholar ↗

2024

Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data

Alexander Havrilla, Wenjing Liao

NeurIPS 2024poster

When training deep neural networks, a model's generalization error is often observed to follow a power scaling law dependent both on the model size and the data size. Perhaps the best known example of such scaling laws are for transformer-based large language models (**LLMs**), where networks with b…

Cited by 10 Source ↗PDF ↗code ↗Scholar ↗

2023

trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback

Alexander Havrilla, Maksym Zhuravinskyi, Duy Van Phung, Aman Tiwari, Jonathan Tow, Stella Biderman +2

EMNLP 2023long main

Reinforcement learning from human feedback (\textbf{RLHF}) utilizes human feedback to better align large language models with human preferences via online optimization against a learned reward model. Current RLHF paradigms rely on Proximal Policy Optimization (\textbf{PPO}), which quickly becomes a…

Cited by 0 Source ↗Scholar ↗

2022

On Deep Generative Models for Approximation and Estimation of Distributions on Manifolds

Biraj Dahal, Alexander Havrilla, Minshuo Chen, Tuo Zhao, Wenjing Liao

NeurIPS 2022accept

Deep generative models have experienced great empirical successes in distribution learning. Many existing experiments have demonstrated that deep generative networks can efficiently generate high-dimensional complex data from a low-dimensional easy-to-sample distribution. However, this phenomenon ca…

Cited by 12 Source ↗PDF ↗Scholar ↗