Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMs
Factual hallucinations are a major challenge for Large Language Models (LLMs). They undermine reliability and user trust by generating inaccurate or fabricated content. Recent studies suggest that when generating false statements, the internal states of LLMs encode information about truthfulness. Ho…