Hongao Zhu

The Inverse Scaling Effect of Pre-Trained Language Model Surprisal Is Not Due to Data Leakage

In psycholinguistic modeling, surprisal from larger pre-trained language models has been shown to be a poorer predictor of naturalistic human reading times. However, it has been speculated that this may be due to data leakage that caused language models to see the text stimuli during training. This…