2024
In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
NeurIPS 2024spotlight
A striking property of transformers is their ability to perform in-context learning (ICL), a machine learning framework in which the learner is presented with a novel context during inference implicitly through some data, and tasked with making a prediction in that context. As such, that learner mus…