Gukhyeon Lee

KnowProxy: Adapting Large Language Models by Knowledge-guided Proxy

Adapting large language models (LLMs) using smaller proxy models has been shown to improve training efficiency, where the LLMs remain frozen while the proxies are tuned on top. However, this approach typically requires access to the output probability distributions of LLMs, which are often inaccessi…