Yizhou Xu — accepted AI-conference papers

2026

Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime

Leonardo Defilippis, Yizhou Xu, Julius Girardin, Vittorio Erba, Emanuele Troiani, Lenka Zdeborová +2

ICLR 2026oral

Neural scaling laws underlie many of the recent advances in deep learning, yet their theoretical understanding remains largely confined to linear models. In this work, we present a systematic analysis of scaling laws for quadratic and diagonal neural networks in the feature learning regime. Leveragi…

Cited by 16 Source ↗code ↗Scholar ↗

2026

Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws

Fabrizio Boncoraglio, Vittorio Erba, Emanuele Troiani, Yizhou Xu, FLORENT KRZAKALA, Lenka Zdeborova

ICML 2026spotlight

Trained attention layers exhibit striking and reproducible spectral structure of the weights, including low-rank collapse, bulk deformation, and isolated spectral outliers, yet the origin of these phenomena and their implications for generalization remain poorly understood. We study empirical risk m…

Cited by 0 Source ↗Scholar ↗

2025

Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions

Yizhou Xu, Florent Krzakala, Lenka Zdeborova

NeurIPS 2025poster

The Restricted Boltzmann Machine (RBM) is one of the simplest generative neural networks capable of learning input distributions. Despite its simplicity, the analysis of its performance in learning from the training data is only well understood in cases that essentially reduce to singular value deco…

Cited by 0 Source ↗code ↗Scholar ↗

2025

Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning

Liu Ziyin, Yizhou Xu, Isaac L. Chuang

NeurIPS 2025poster

With the rapid discovery of emergent phenomena in deep learning and large language models, understanding their cause has become an urgent need. Here, we propose a rigorous entropic-force theory for understanding the learning dynamics of neural networks trained with stochastic gradient descent (SGD)…

Cited by 0 Source ↗Scholar ↗

2025

Remove Symmetries to Control Model Expressivity and Improve Optimization

Liu Ziyin, Yizhou Xu, Isaac L. Chuang

ICLR 2025poster

When symmetry is present in the loss function, the model is likely to be trapped in a low-capacity state that is sometimes known as a ``collapse." Being trapped in these low-capacity states can be a major obstacle to training across many scenarios where deep learning technology is applied. We first…

Cited by 0 Source ↗PDF ↗Scholar ↗

2025

Three Mechanisms of Feature Learning in a Linear Network

Yizhou Xu, Liu Ziyin

ICLR 2025poster

Understanding the dynamics of neural networks in different width regimes is crucial for improving their training and performance. We present an exact solution for the learning dynamics of a one-hidden-layer linear network, with one-dimensional data, across any finite width, uniquely exhibiting both…

Cited by 0 Source ↗PDF ↗Scholar ↗

2024

Continual Driving Policy Optimization with Closed-Loop Individualized Curricula

Haoyi Niu, Yizhou Xu, Xingjian Jiang, Jianming Hu

ICRA 2024poster

The safety of autonomous vehicles (AV) has been a long-standing top concern, stemming from the absence of rare and safety-critical scenarios in the long-tail naturalistic driving distribution. To tackle this challenge, a surge of research in scenario-based autonomous driving has emerged, with a focu…

Cited by 3 Source ↗code ↗Scholar ↗

2024

Decentralizing Coherent Joint Transmission Precoding Via Deterministic Equivalents

Yuhao Liu, Xinyu Bian, Yizhou Xu, Tianqi Hou, Wenjie Wang, Yuyi Mao +1

ICASSP 2024accepted

In order to control the inter-cell interference for a multi-cell multi-user multiple-input multiple-output network, we consider the precoder design for coordinated multi-point with downlink coherent joint transmission. To avoid costly information exchange among the cooperating base stations in a cen…

Cited by 0 Source ↗Scholar ↗