Avital Oliver — accepted AI-conference papers

2023

Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution

Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron +9

NeurIPS 2023poster

The ubiquitous and demonstrably suboptimal choice of resizing images to a fixed resolution before processing them with computer vision models has not yet been successfully challenged. However, models such as the Vision Transformer (ViT) offer flexible sequence-based modeling, and hence varying input…

Cited by 108 Source ↗PDF ↗Scholar ↗

2023

Scaling Vision Transformers to 22 Billion Parameters

Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer +36

ICML 2023oral

The scaling of Transformers has driven breakthrough capabilities for language models. At present, the largest large language models (LLMs) contain upwards of 100B parameters. Vision Transformers (ViT) have introduced the same architecture to image and video modelling, but these have not yet been suc…

Cited by 650 Source ↗PDF ↗Scholar ↗

2019

MixMatch: A Holistic Approach to Semi-Supervised Learning

David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin A Raffel

NeurIPS 2019poster

Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets. In this work, we unify the current dominant approaches for semi-supervised learning to produce a new algorithm, MixMatch, that guesses low-entropy labels fo…

Cited by 4153 Source ↗PDF ↗code ↗Scholar ↗

2019

S4L: Self-Supervised Semi-Supervised Learning

Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, Lucas Beyer

ICCV 2019oral

This work tackles the problem of semi-supervised learning of image classifiers. Our main insight is that the field of semi-supervised learning can benefit from the quickly advancing field of self-supervised visual representation learning. Unifying these two approaches, we propose the framework of se…

Cited by 864 PDF ↗Scholar ↗

2018

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms

Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, Ian Goodfellow

NeurIPS 2018spotlight

Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, we argue that these benchmarks fail to address…

Cited by 1259 Source ↗PDF ↗code ↗Scholar ↗