Po-Chiao Lin

Universality, Function Composition, and Algorithm Emulation All In-Context

We study the in-context universal approximation and compositional generalization of softmax Transformers. We prove an in-context universality result: a fixed-weight softmax Transformer approximates a broad class of continuous sequence-to-sequence functions. Building on this universality, we establis…