Thanawat Lodkaew

How Can I Publish My LLM Benchmark Without Giving the True Answers Away?

Publishing a large language model (LLM) benchmark (especially its ground-truth answers) on the Internet risks contaminating future LLMs and enabling evaluation gaming: it may be unintentionally (or intentionally) used to train or select a model, or exploited to overfit and hack leaderboards when lab…