2025
How Memory in Optimization Algorithms Implicitly Modifies the Loss
NeurIPS 2025poster
In modern optimization methods used in deep learning, each update depends on the history of previous iterations, often referred to as memory, and this dependence decays fast as the iterates go further into the past. For example, gradient descent with momentum has exponentially decaying memory throug…