It Ain't Over: A Multi-aspect Diverse Math Word Problem Dataset
Jiwoo Kim, Youngbin Kim, Ilwoong Baek, JinYeong Bak, Jongwuk Lee
Abstract
The math word problem (MWP) is a complex task that requires natural language understanding and logical reasoning to extract key knowledge from natural language narratives. Previous studies have provided various MWP datasets but lack diversity in problem types, lexical usage patterns, languages, and annotations for intermediate solutions. To address these limitations, we introduce a new MWP dataset, named DMath (Diverse Math Word Problems), offering a wide range of diversity in problem types, lexical usage patterns, languages, and intermediate solutions. The problems are available in English and Korean and include an expression tree and Python code as intermediate solutions. Through extensive experiments, we demonstrate that the DMath dataset provides a new opportunity to evaluate the capability of large language models, i.e., GPT-4 only achieves about 75\% accuracy on the DMath dataset.
BibTeX
@inproceedings{
kim2023it,
title={It Ain't Over: A Multi-aspect Diverse Math Word Problem Dataset},
author={Jiwoo Kim and Youngbin Kim and Ilwoong Baek and JinYeong Bak and Jongwuk Lee},
booktitle={The 2023 Conference on Empirical Methods in Natural Language Processing},
year={2023},
url={https://openreview.net/forum?id=3LIUMrCKrv}
}