Abhiti Mishra

Continuum Transformers Perform In-Context Learning by Operator Gradient Descent

Transformers robustly exhibit the ability to perform in-context learning, whereby their predictive accuracy on a task can increase not by parameter updates but merely with the placement of training samples in their context windows. Recent works have shown that transformers achieve this by implementi…