Zelong Guan

Practical Offloading for Fine-Tuning LLM on Commodity GPU via Learned Sparse Projectors

Fine-tuning large language models (LLMs) requires significant memory, often exceeding the capacity of a single GPU. A common solution to this memory challenge is offloading compute and data from the GPU to the CPU. However, this approach is hampered by the limited bandwidth of commodity hardware, wh…