LMCodec: A Low Bitrate Speech Codec with Causal Transformer Models
Teerapat Jenrungrot, Michael Chinen, W. Bastiaan Kleijn, Jan Skoglund, Zalán Borsos, Neil Zeghidour, Marco Tagliasacchi
Abstract
We introduce LMCodec, a causal neural speech codec that provides high quality audio at very low bitrates. The backbone of the system is a causal convolutional codec that encodes audio into a hierarchy of coarse-to-fine tokens using residual vector quantization. LMCodec trains a Transformer language model to predict the fine tokens from the coarse ones in a generative fashion, allowing for the transmission of fewer codes. A second Transformer predicts the uncertainty of the next codes given the past transmitted codes, and is used to perform conditional entropy coding. A MUSHRA subjective test was conducted and shows that the quality is comparable to reference codecs at higher bitrates. Example audio is available at https://mjenrungrot.github.io/chrome-media-audio-papers/publications/lmcodec.
BibTeX
@inproceedings{icassp2023_lmcodecalowbitra,
title = {LMCodec: A Low Bitrate Speech Codec with Causal Transformer Models},
author = {Teerapat Jenrungrot and Michael Chinen and W. Bastiaan Kleijn and Jan Skoglund and Zalán Borsos and Neil Zeghidour and Marco Tagliasacchi},
booktitle = {ICASSP 2023},
year = {2023}
}