Does Knowing When Help Is Needed Improve Subgoal Hint Performance in an Intelligent Data-Driven Logic Tutor?
Nazia Alam, Mehak Maniktala, Behrooz Mostafavi, Min Chi, Tiffany Barnes
Abstract
The assistance dilemma is a well-recognized challenge to determine when and how to provide help during problem solving in intelligent tutoring systems. This dilemma is particularly challenging to address in domains such as logic proofs, where problems can be solved in a variety of ways. In this study, we investigate two data-driven techniques to address the when and how of the assistance dilemma, combining a model that predicts when students need help learning efficient strategies, and hints that suggest what subgoal to achieve. We conduct a study assessing the impact of the new pedagogical policy against a control policy without these adaptive components. We found empirical evidence which suggests that showing subgoals in training problems upon predictions of the model helped the students who needed it most and improved test performance when compared to their control peers. Our key findings include significantly fewer steps in posttest problem solutions for students with low prior proficiency and significantly reduced help avoidance for all students in training.
BibTeX
@article{Alam_Maniktala_Mostafavi_Chi_Barnes_2024, title={Does Knowing When Help Is Needed Improve Subgoal Hint Performance in an Intelligent Data-Driven Logic Tutor?}, volume={37}, url={https://ojs.aaai.org/index.php/AAAI/article/view/26887}, DOI={10.1609/aaai.v37i13.26887}, abstractNote={The assistance dilemma is a well-recognized challenge to determine
when and how to provide help during problem solving
in intelligent tutoring systems. This dilemma is particularly
challenging to address in domains such as logic proofs,
where problems can be solved in a variety of ways. In this
study, we investigate two data-driven techniques to address
the when and how of the assistance dilemma, combining a
model that predicts when students need help learning efficient
strategies, and hints that suggest what subgoal to achieve.
We conduct a study assessing the impact of the new pedagogical
policy against a control policy without these adaptive
components. We found empirical evidence which suggests
that showing subgoals in training problems upon predictions
of the model helped the students who needed it most
and improved test performance when compared to their control
peers. Our key findings include significantly fewer steps
in posttest problem solutions for students with low prior proficiency
and significantly reduced help avoidance for all students
in training.}, number={13}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Alam, Nazia and Maniktala, Mehak and Mostafavi, Behrooz and Chi, Min and Barnes, Tiffany}, year={2024}, month={Jul.}, pages={15895-15902} }