Shihui Song

Exposing and Exploiting Fine-Grained Block Structures for Fast and Accurate Sparse Training

Sparse training is a popular technique to reduce the overhead of training large models. Although previous work has shown promising results for nonstructured sparse models, it is still unclear whether a sparse model with structural constraints can be trained from scratch to high accuracy. In this wor…