Anxin Guo

Hallucination is a Consequence of Space-Optimality: A Rate-Distortion Theorem for Membership Testing

Large language models often hallucinate with high confidence on "random facts" that lack inferable patterns. We formalize the memorization of such facts as a membership testing problem, unifying the discrete error metrics of Bloom filters with the continuous log-loss of LLMs. By analyzing this probl…