AIConfPaper
← Search

Ari S Morcos

15 accepted papers

2024

Effective pruning of web-scale datasets based on complexity of concept clusters

Amro Kamal Mohamed Abbas, Evgenia Rusak, Kushal Tirumala, Wieland Brendel, Kamalika Chaudhuri, Ari S. Morcos
ICLR 2024poster

Utilizing massive web-scale datasets has led to unprecedented performance gains in machine learning models, but also imposes outlandish compute requirements for their training. In order to improve training and data efficiency, we here push the limits of pruning large-scale multimodal datasets for tr…

Cited by 26Source ↗PDF ↗code ↗Scholar ↗
2024

Sieve: Multimodal Dataset Pruning using Image Captioning Models

Anas Mahmoud, Mostafa Elhoushi, Amro Abbas, Yu Yang, Newsha Ardalani, Hugh Leather +1
CVPR 2024poster

Vision-Language Models (VLMs) are pretrained on large diverse and noisy web-crawled datasets. This underscores the critical need for dataset pruning as the quality of these datasets is strongly correlated with the performance of VLMs on downstream tasks. Using CLIPScore from a pretrained model to on…

Cited by 22Source ↗PDF ↗code ↗Scholar ↗
2023

Emergence of Maps in the Memories of Blind Navigation Agents

Erik Wijmans, Manolis Savva, Irfan Essa, Stefan Lee, Ari S. Morcos, Dhruv Batra
ICLR 2023top-5%

Animal navigation research posits that organisms build and maintain internal spa- tial representations, or maps, of their environment. We ask if machines – specifically, artificial intelligence (AI) navigation agents – also build implicit (or ‘mental’) maps. A positive answer to this question would…

Cited by 31Source ↗PDF ↗Scholar ↗
2023

PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning

Florian Bordes, Shashank Shekhar, Mark Ibrahim, Diane Bouchacourt, Pascal Vincent, Ari S. Morcos
NeurIPS 2023poster

Synthetic image datasets offer unmatched advantages for designing and evaluating deep neural networks: they make it possible to (i) render as many data samples as needed, (ii) precisely control each scene and yield granular ground truth labels (and captions), (iii) precisely control distribution shi…

Cited by 18Source ↗PDF ↗code ↗Scholar ↗
2023

Stable and low-precision training for large-scale vision-language models

Mitchell Wortsman, Tim Dettmers, Luke Zettlemoyer, Ari S. Morcos, Ali Farhadi, Ludwig Schmidt
NeurIPS 2023poster

We introduce new methods for 1) accelerating and 2) stabilizing training for large language-vision models. 1) For acceleration, we introduce SwitchBack, a linear layer for int8 quantized training which provides a speed-up of 13-25% while matching the performance of bfloat16 training within 0.1 perce…

Cited by 44Source ↗PDF ↗code ↗Scholar ↗
2022

Beyond neural scaling laws: beating power law scaling via data pruning

Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari S. Morcos
NeurIPS 2022accept

Widely observed neural scaling laws, in which error falls off as a power of the training set size, model size, or both, have driven substantial performance improvements in deep learning. However, these improvements through scaling alone require considerable costs in compute and energy. Here we focus…

Cited by 445Source ↗PDF ↗code ↗Scholar ↗
2022

COAT: Measuring Object Compositionality in Emergent Representations

Sirui Xie, Ari S Morcos, Song-Chun Zhu, Ramakrishna Vedantam
ICML 2022spotlight

Learning representations that can decompose a multi-object scene into its constituent objects and recompose them flexibly is desirable for object-oriented reasoning and planning. Built upon object masks in the pixel space, existing metrics for objectness can only evaluate generative models with an o…

Cited by 9Source ↗PDF ↗Scholar ↗
2022

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos +5
ICML 2022spotlight

The conventional recipe for maximizing model accuracy is to (1) train multiple models with various hyperparameters and (2) pick the individual model which performs best on a held-out validation set, discarding the remainder. In this paper, we revisit the second step of this procedure in the context…

Cited by 1132Source ↗PDF ↗code ↗Scholar ↗
2021

ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

Stéphane D’Ascoli, Hugo Touvron, Matthew L Leavitt, Ari S Morcos, Giulio Biroli, Levent Sagun
ICML 2021spotlight

Convolutional architectures have proven extremely successful for vision tasks. Their hard inductive biases enable sample-efficient learning, but come at the cost of a potentially lower performance ceiling. Vision Transformers (ViTs) rely on more flexible self-attention layers, and have recently outp…

Cited by 1025Source ↗PDF ↗code ↗Scholar ↗
2021

Grounding inductive biases in natural images: invariance stems from variations in data

Diane Bouchacourt, Mark Ibrahim, Ari S. Morcos
NeurIPS 2021poster

To perform well on unseen and potentially out-of-distribution samples, it is desirable for machine learning models to have a predictable response with respect to transformations affecting the factors of variation of the input. Here, we study the relative importance of several types of inductive bias…

Cited by 28Source ↗PDF ↗code ↗Scholar ↗
2021

Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs

Matthew L Leavitt, Ari S. Morcos
ICLR 2021poster

The properties of individual neurons are often analyzed in order to understand the biological and artificial neural networks in which they're embedded. Class selectivity—typically defined as how different a neuron's responses are across different classes of stimuli or data samples—is commonly used f…

Cited by 58Source ↗PDF ↗Scholar ↗
2021

Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs

Jonathan Frankle, David J. Schwab, Ari S. Morcos
ICLR 2021poster

A wide variety of deep learning techniques from style transfer to multitask learning rely on training affine transformations of features. Most prominent among these is the popular feature normalization technique BatchNorm, which normalizes activations and then subsequently applies a learned affine t…

Cited by 153Source ↗PDF ↗Scholar ↗
2020

Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP

Haonan Yu, Sergey Edunov, Yuandong Tian, Ari S. Morcos
ICLR 2020poster

The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a “lucky” sub-network initialization being present rather than by helping the optimization process (Frankle& Carbin, 2019). Intriguingly, this phenomenon su…

Cited by 157Source ↗Scholar ↗
2018

On the importance of single directions for generalization

Ari S. Morcos, David G.T. Barrett, Neil C. Rabinowitz, Matthew Botvinick
ICLR 2018poster

Despite their ability to memorize large datasets, deep neural networks often achieve good generalization performance. However, the differences between the learned solutions of networks which generalize and those which do not remain unclear. Additionally, the tuning properties of single directions (d…

Cited by 388Source ↗PDF ↗Scholar ↗