← Search

An Li

2 accepted papers

2025

Periodical Moving Average Accelerates Gradient Accumulation for Post-Training

UAI 2025

High gradient variance presents a significant obstacle to efficient post-training of large language models (LLMs) on memory-constrained devices. Existing practical strategies-such as reducing batch sizes or adopting gradient accumulation (GA)-suffer from an inherent trade-off: smaller batches exacer

Cited by 0SourcePDFScholar