Jiali Pang

Acceleration of Large Transformer Model Training by Sensitivity-Based Layer Dropping

Transformer models are widely used in AI applications such as Natural Language Processing (NLP), Computer Vision (CV), etc. However, enormous computation workload be-comes an obstacle to train large transformer models efficiently. Recently, some methods focus on reducing the computation workload dur…