Nayeema Nonta

SubTrack++ : Gradient Subspace Tracking for Scalable LLM Training

Training large language models (LLMs) is highly resource-intensive due to their massive number of parameters and the overhead of optimizer states. While recent work has aimed to reduce memory consumption, such efforts often entail trade-offs among memory efficiency, training time, and model performa…