Analyzing and Mitigating Inconsistency in Discrete Speech Tokens for Neural Codec Language Models
Building upon advancements in Large Language Models (LLMs), the field of audio processing has seen increased interest in training speech generation tasks with discrete speech token sequences. However, directly discretizing speech by neural audio codecs often results in sequences that fundamentally d…