Llukman Cerkezi

KOALA: A Kalman Optimization Algorithm with Loss Adaptivity

Optimization is often cast as a deterministic problem, where the solution is found through some iterative procedure such as gradient descent. However, when training neural networks the loss function changes over (iteration) time due to the randomized selection of a subset of the samples. This random…