Alireza Amiri Bavandpour

Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers

Chain-of-thought reasoning and scratchpads have emerged as critical tools for enhancing the computational capabilities of transformers. While theoretical results show that polynomial-length scratchpads can extend transformers' expressivity from $TC^0$ to $PTIME$, their required length remains poorly…