Mary Letey

2 accepted papers

2026

Pretrain–Test Task Alignment Governs Generalization in In-Context Learning

Mary Letey, Jacob A Zavatone-Veth, Yue M. Lu, Cengiz Pehlevan

ICLR 2026poster

In-context learning (ICL) is a central capability of Transformer models, but the structures in data that enable its emergence and govern its robustness remain poorly understood. In this work, we study how the structure of pretraining tasks governs generalization in ICL. Using a solvable model for IC…

Cited by 0 Source ↗Scholar ↗

2026

Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time

Blake Bordelon, Mary Letey, Cengiz Pehlevan

ICLR 2026poster

We study in-context learning (ICL) of linear regression in a deep linear self-attention model, characterizing how performance depends on various computational and statistical resources (width, depth, number of training steps, batch size and data per context). In a joint limit where data dimension, c…

Cited by 0 Source ↗Scholar ↗