Liron Soffer

From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning

Humans organize knowledge into compact categories that balance compression with semantic meaning preservation. Large Language Models (LLMs) demonstrate striking linguistic abilities, yet whether they achieve this same balance remains unclear. We apply the Information Bottleneck principle to quantita…