2025
Systematic Generalization in Language Models Scales with Information Entropy
ACL 2025finding
Systematic generalization remains challenging for current language models, which are known to be both sensitive to semantically similar permutations of the input and to struggle with known concepts presented in novel contexts. Although benchmarks exist for assessing compositional behavior, it is unc…