2022
On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model
NAACL 2022long
Many recent studies on large-scale language models have reported successful in-context zero- and few-shot learning ability. However, the in-depth analysis of when in-context learning occurs is still lacking. For example, it is unknown how in-context learning performance changes as the training corpu…