What Do Large Language Models Know About Opinions?
What large language models (LLMs) know about human opinions has important implications for aligning LLMs to human values, simulating humans with LLMs, and understanding what LLMs learn during training. While prior works have tested LLMs' knowledge of opinions via their next token outputs, we present…