Melody Lo

U-shaped and Inverted-U Scaling behind Emergent Abilities of Large Language Models

Large language models (LLMs) have been shown to exhibit *emergent abilities* in some downstream tasks, where model performance stagnates at first and then improves sharply and unpredictably with scale beyond a threshold. In this work, we investigate the phenomenon by grouping questions based on diff…