Mikey Shechter

Filter Like You Test: Data-Driven Data Filtering for CLIP Pretraining

We introduce Filter Like You Test (FLYT), an algorithm for curating large-scale vision-language datasets that *learns* the usefulness of each data point as a pretraining example. FLYT trains a scoring model that learns to weigh each example's features using gradient signals from downstream tasks tra…