Low Bitrate Loss Resilience Scheme for a Speech Enhancing Neural Codec

Abstract

Deep neural networks have proven their efficacy in encoding high-quality speech and audio at remarkably low bitrates, while also demonstrating superior performance in audio packet loss concealment (PLC) compared to traditional methods. Although ultra low-bitrate speech and audio codecs may appear less practical for real-time voice communication over the Internet due to packetization overhead, they present a promising solution for ensuring uninterrupted voice communication under adverse network conditions. In this paper, we use a neural speech codec designed end-to-end, encompassing a versatile set of features ranging from efficient low-bitrate speech coding and decoding to advanced functionalities such as noise removal, dereverberation, and packet loss concealment. For this codec, we present a long low-bitrate redundancy mechanism for recovering from extended packet loss bursts. We furthermore introduce a memory-efficient entropy coding scheme specifically designed for low-bitrate redundant audio packets. Finally, we demonstrate the effectiveness of the said codec, together with the memory- and bitrate-efficient redundancy, at coping with adverse acoustic and network conditions.