2024
Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
NeurIPS 2024poster
Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization, we introduce a subtle modification to the next-token training objective that we call the goldfish loss. During training, a randomly sampled subsets of tokens are excluded…