Byeongchan Lee — accepted AI-conference papers

2025

Peri-LN: Revisiting Normalization Layer in the Transformer Architecture

Jeonghoon Kim, Byeongchan Lee, Cheonbok Park, Yeontaek Oh, Beomjun Kim, Taehwan Yoo +4

ICML 2025poster

Selecting a layer normalization (LN) strategy that stabilizes training and speeds convergence in Transformers remains difficult, even for today’s large language models (LLM). We present a comprehensive analytical foundation for understanding how different LN strategies influence training dynamics in…

Cited by 0 Source ↗PDF ↗Scholar ↗

2023

Implicit Contrastive Representation Learning with Guided Stop-gradient

Byeongchan Lee, Sehyun Lee

NeurIPS 2023poster

In self-supervised representation learning, Siamese networks are a natural architecture for learning transformation-invariance by bringing representations of positive pairs closer together. But it is prone to collapse into a degenerate solution. To address the issue, in contrastive learning, a contr…

Cited by 1 Source ↗PDF ↗code ↗Scholar ↗