vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations

Alexei Baevski; Steffen Schneider; Michael Auli

ICLR 2020poster811 citations

vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations

Alexei Baevski, Steffen Schneider, Michael Auli

Abstract

We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task. The algorithm uses either a gumbel softmax or online k-means clustering to quantize the dense representations. Discretization enables the direct application of algorithms from the NLP community which require discrete inputs. Experiments show that BERT pre-training achieves a new state of the art on TIMIT phoneme classification and WSJ speech recognition.

speech recognitionspeech representation learning

BibTeX

@inproceedings{
Baevski2020vq-wav2vec:,
title={vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations},
author={Alexei Baevski and Steffen Schneider and Michael Auli},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://openreview.net/forum?id=rylwJxrYDS}
}