Chris Köcher — accepted AI-conference papers

2026

The Counting Power of Transformers

Marco Sälzer, Chris Köcher, Alexander Kozachinskiy, Georg Zetzsche, Anthony Widjaja Lin

ICLR 2026poster

Counting properties (e.g. determining whether certain tokens occur more than other tokens in a given input text) have played a significant role in the study of expressiveness of transformers. In this paper, we provide a formal framework for investigating the counting power of trans…

Cited by 5 Source ↗Scholar ↗

2024

The Power of Hard Attention Transformers on Data Sequences: A formal language theoretic perspective

Pascal Bergsträßer, Chris Köcher, Anthony Widjaja Lin, Georg Zetzsche

NeurIPS 2024poster

Formal language theory has recently been successfully employed to unravel the power of transformer encoders. This setting is primarily applicable in Natural Language Processing (NLP), as a token embedding function (where a bounded number of tokens is admitted) is first applied before fe…

Cited by 1 Source ↗PDF ↗Scholar ↗