2024
CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization
NeurIPS 2024poster
Training large AI models such as LLMs and DLRMs costs massive GPUs and computing time. The high training cost has become only affordable to big tech companies, meanwhile also causing increasing concerns about the environmental impact. This paper presents CoMERA, a **Co**mputing- and **M**emory-**E**…