AAAI 2025technical0 citations
Using Next Sentence Prediction to Test ChatGPT’s Text Comprehension (Student Abstract)
Ojas M Agarwal, Madelein Villegas, Jack Mostow
Abstract
We propose the Next Sentence Prediction (NSP) task as a simple, objective, scalable, automated way to test ChatGPT’s text comprehension. Given a context excerpted from a children’s story, the task is to distinguish the next story sentence from a later sentence in the story. We analyze how ChatGPT’s performance on this task is related to various features of the text, using data from English and Swahili children’s stories.
BibTeX
@article{Agarwal_Villegas_Mostow_2025, title={Using Next Sentence Prediction to Test ChatGPT’s Text Comprehension (Student Abstract)}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/35238}, DOI={10.1609/aaai.v39i28.35238}, abstractNote={We propose the Next Sentence Prediction (NSP) task as a simple, objective, scalable, automated way to test ChatGPT’s text comprehension. Given a context excerpted from a children’s story, the task is to distinguish the next story sentence from a later sentence in the story. We analyze how ChatGPT’s performance on this task is related to various features of the text, using data from English and Swahili children’s stories.}, number={28}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Agarwal, Ojas M and Villegas, Madelein and Mostow, Jack}, year={2025}, month={Apr.}, pages={29322-29324} }