Mayank Shrivastava

Sketching for Distributed Deep Learning: A Sharper Analysis

The high communication cost between the server and the clients is a significant bottleneck in scaling distributed learning for overparametrized deep models. One popular approach for reducing this communication overhead is randomized sketching. However, existing theoretical analyses for sketching-bas…