2023
Dynamic Stashing Quantization for Efficient Transformer Training
EMNLP 2023short findings
Large Language Models (LLMs) have demonstrated impressive performance on a range of Natural Language Processing (NLP) tasks. Unfortunately, the immense amount of computations and memory accesses required for LLM training makes them prohibitively expensive in terms of hardware cost, and thus challeng…