Anton Sugolov

2 accepted papers

2026

Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimization

Haoming Meng, Anton Sugolov, Vardan Papyan

ICML 2026poster

Deep neural networks with repeated blocks, such as transformers and ResNets, often exhibit closely related representational structure across layers that emerges with training. Motivated by this observation, we introduce *Gradient Smoothing*, a general training paradigm that couples gradient updates …

Cited by 0 Source ↗Scholar ↗

2025

Transformer Block Coupling and its Correlation with Generalization in LLMs

Murdock Aubry, Haoming Meng, Anton Sugolov, Vardan Papyan

ICLR 2025poster

Large Language Models (LLMs) have made significant strides in natural language processing, and a precise understanding of the internal mechanisms driving their success is essential. In this work, we analyze the trajectories of token embeddings as they pass through transformer blocks, linearizing the…

Cited by 0 Source ↗PDF ↗Scholar ↗