Alexander Mordvintsev — accepted AI-conference papers

2023

Transformers Learn In-Context by Gradient Descent

Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, Joao Sacramento, Alexander Mordvintsev, Andrey Zhmoginov +1

ICML 2023oral

At present, the mechanisms of in-context learning in Transformers are not well understood and remain mostly an intuition. In this paper, we suggest that training Transformers on auto-regressive objectives is closely related to gradient-based meta-learning formulations. We start by providing a simple…

Cited by 523 Source ↗PDF ↗code ↗Scholar ↗

2017

Associative Domain Adaptation

Philip Haeusser, Thomas Frerix, Alexander Mordvintsev, Daniel Cremers

ICCV 2017poster

We propose "associative domain adaptation", a novel technique for end-to-end domain adaptation with neural networks, the task of inferring class labels for an unlabeled target domain based on the statistical properties of a labeled source domain. Our training scheme follows the paradigm that in orde…

Cited by 10 PDF ↗Scholar ↗

2017

Learning by Association -- A Versatile Semi-Supervised Training Method for Neural Networks

Philip Haeusser, Alexander Mordvintsev, Daniel Cremers

CVPR 2017poster

In many real-world scenarios, labeled data for a specific machine learning task is costly to obtain. Semi-supervised training methods make use of abundantly available unlabeled data and a smaller number of labeled examples. We propose a new framework for semi-supervised training of deep neural netwo…

Cited by 153 PDF ↗Scholar ↗