Dan Pechi

Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale.

In recent years, language models have drastically grown in size, and the abilities of these models have been shown to improve with scale. The majority of recent scaling laws studies focused on high-compute high-parameter count settings, leaving the question of when these abilities begin to emerge la…