Yanming Lai

Approximation Bounds for Transformer Networks with Application to Regression

We develop approximation and statistical theory for standard Transformer networks in sequence modeling. Given a sequence-to-sequence target on $[0,1]^{d_x \times n}$ whose entries are $\gamma$-H\"older for $\gamma \in (0,1]$ or belong to a first-order Sobolev class, we establish explicit $L^p$-appro…