CROSSWORD: A Semantic Approach To Text Compression Via Masking

Abstract

Conventional data compression methods typically model the information source as an i.i.d. stochastic process, thereby establishing the fundamental limit as entropy for lossless compression and as mutual information for lossy compression. However, the source in the real world (e.g., text, music, and speech) is often statistically ill-defined because of its close connection to human perception. This work aims to exploit the semantic aspect of text as inspired by the puzzle crossword. The main idea is to only compress those semantically important words while masking the rest; the proposed decompressor can recover all the missing words automatically according to context. Experiments show that the proposed semantic approach can achieve much higher compression efficiency than the state-of-the-art semantic compression method.