Rachael Tappenden

SONIA: A Symmetric Blockwise Truncated Optimization Algorithm

This work presents a new optimization algorithm for empirical risk minimization. The algorithm bridges the gap between first- and second-order methods by computing a search direction that uses a second-order-type update in one subspace, coupled with a scaled steepest descent step in the orthogonal c…