Hasan Abed Al Kader Hammoud*

On Pretraining Data Diversity for Self-Supervised Learning

"We explore the impact of training with more diverse datasets, characterized by the number of unique samples, on the performance of self-supervised learning (SSL) under a fixed computational budget. Our findings demonstrate that increasing pretraining data diversity enhances SSL performance, albeit…