2024
Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning
ACL 2024findings
Neural Machine Translation models are extremely data and compute-hungry. However, not all datapoints contribute equally to model training and generalization. Data pruning to remove the low-value data points has the benefit of drastically reducing the compute budget without significantdrop in model p…