2023
Large language models transition from integrating across position-yoked, exponential windows to structure-yoked, power-law windows
NeurIPS 2023poster
Modern language models excel at integrating across long temporal scales needed to encode linguistic meaning and show non-trivial similarities to biological neural systems. Prior work suggests that human brain responses to language exhibit hierarchically organized "integration windows" that substanti…