Kevin Christian Wibisono

From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When

Large language models (LLMs) like transformers demonstrate impressive in-context learning (ICL) capabilities, allowing them to make predictions for new tasks based on prompt exemplars without parameter updates. While existing ICL theories often assume structured training data resembling ICL tasks (e…