Xingyu Lyu — accepted AI-conference papers

2026

The Devil is in the Condition Numbers: Why is GLU Better than non-GLU Structure?

Xingyu Lyu, Qianqian Xu, zhiyong yang, Peisong Wen, Qingming Huang

ICML 2026poster

Gated Linear Units (GLU) and their variants are widely adopted in modern open-source large language model architectures and consistently outperform their non-gated counterparts, yet the underlying reasons for this advantage remain unclear. In this work, we study GLU by analyzing two-layer networks i…

Cited by 0 Source ↗Scholar ↗

2025

SSE-SAM: Balancing Head and Tail Classes Gradually Through Stage-Wise SAM

Xingyu Lyu, Qianqian Xu, Zhiyong Yang, Shaojie Lyu, Qingming Huang

AAAI 2025technical

Real-world datasets often exhibit a long-tailed distribution, where vast majority of classes known as tail classes have only few samples. Traditional methods tend to overfit on these tail classes. Recently, a new approach called Imbalanced SAM (ImbSAM) is proposed to leverage the generalization bene…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗