Kyungmin Bin

ASPEN: Breaking Operator Barriers for Efficient Parallelization of Deep Neural Networks

Modern Deep Neural Network (DNN) frameworks use tensor operators as the main building blocks of DNNs. However, we observe that operator-based construction of DNNs incurs significant drawbacks in parallelism in the form of synchronization barriers. Synchronization barriers of operators confine the sc…