Enhancing Teacher Classroom Behavior Descriptions: A Spatio-Temporal Graph-Based Method for Video Captioning

Abstract

Teacher behavior description is an objective record of teachers’ behaviors in the teaching process, aiming to provide evidence for reflection on teaching behavior. Although video captioning technology can automatically generate behavior description, existing research on teacher behavior description is mostly limited to case studies due to the lack of relevant datasets. At the same time, existing video captioning methods cannot effectively capture the dynamic characteristics of teacher behavior in real classrooms. In this paper, we propose a novel spatio-temporal graph neural network (STGN), which represents teacher-student interactions in the spatial domain using action features and models the temporal relationships of teachers guided by contextual information. The model achieves satisfactory results on both the proprietary teacher behavior description (TBD) dataset and two public datasets, MSR-VTT and MSVD, demonstrating the effectiveness of our approach.