Roman Kim

TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings

This paper presents the Text Encoding Diffusion Model (TEncDM), a novel approach to diffusion modeling that operates in the space of pre-trained language model encodings. In contrast to traditionally used embeddings, encodings integrate contextual information. In our approach, we also employ a trans…