Myungheon Chin

PinTok: Tokenizers Deserve Dedicated Pinned CPU-Compute and Memory

Tokenization is the first point of contact between large language models (LLMs) and text data, yet it has not been viewed by many as a component of LLMs worth accelerating. During inference, tokenizers typically rely on simple dictionary lookups and are executed on CPUs as standard processes. This a…