← Search

Jan van den Brand

2 accepted papers

2024

Algorithm and Hardness for Dynamic Attention Maintenance in Large Language Models

ICML 2024poster

The attention scheme is one of the key components over all the LLMs, such as BERT, GPT-1, Transformers, GPT-2, 3, 3.5 and 4. Inspired by previous theoretical study of static version of the attention multiplication problem [Zandieh, Han, Daliri, and Karbasi ICML 2023, Alman and Song NeurIPS 2023], we…

Cited by 37SourcePDFScholar