Gleb Kumichev

On the Way to Lossless Compression of Language Transformers: Exploring Cross-Domain Properties of Quantization

Modern Transformers achieved impressive results on various Natural Language Processing tasks over the last few years. The one downside of this success is the size of these models. Huge capacity, which sometimes surpasses billions of parameters, improves generalization abilities, but makes it difficu…