Jacob Trauger

Sequence Length Independent Norm-Based Generalization Bounds for Transformers

This paper provides norm-based generalization bounds for the Transformer architecture that do not depend on the input sequence length. We employ a covering number based approach to prove our bounds. We use three novel covering number bounds for the function class of bounded linear mappings to upper…