H-AES: Towards Automated Essay Scoring for Hindi
Shubhankar Singh, Anirudh Pupneja, Shivaansh Mital, Cheril Shah, Manish Bawkar, Lakshman Prasad Gupta, Ajit Kumar, Yaman Kumar
Abstract
The use of Natural Language Processing (NLP) for Automated Essay Scoring (AES) has been well explored in the English language, with benchmark models exhibiting performance comparable to human scorers. However, AES in Hindi and other low-resource languages remains unexplored. In this study, we reproduce and compare state-of-the-art methods for AES in the Hindi domain. We employ classical feature-based Machine Learning (ML) and advanced end-to-end models, including LSTM Networks and Fine-Tuned Transformer Architecture, in our approach and derive results comparable to those in the English language domain. Hindi being a low-resource language, lacks a dedicated essay-scoring corpus. We train and evaluate our models using translated English essays and empirically measure their performance on our own small-scale, real-world Hindi corpus. We follow this up with an in-depth analysis discussing prompt-specific behavior of different language models implemented.
BibTeX
@article{Singh_Pupneja_Mital_Shah_Bawkar_Gupta_Kumar_Kumar_Gupta_Ratn Shah_2024, title={H-AES: Towards Automated Essay Scoring for Hindi}, volume={37}, url={https://ojs.aaai.org/index.php/AAAI/article/view/26894}, DOI={10.1609/aaai.v37i13.26894}, abstractNote={The use of Natural Language Processing (NLP) for Automated Essay Scoring (AES) has been well explored in the English language, with benchmark models exhibiting performance comparable to human scorers. However, AES in Hindi and other low-resource languages remains unexplored. In this study, we reproduce and compare state-of-the-art methods for AES in the Hindi domain. We employ classical feature-based Machine Learning (ML) and advanced end-to-end models, including LSTM Networks and Fine-Tuned Transformer Architecture, in our approach and derive results comparable to those in the English language domain. Hindi being a low-resource language, lacks a dedicated essay-scoring corpus. We train and evaluate our models using translated English essays and empirically measure their performance on our own small-scale, real-world Hindi corpus. We follow this up with an in-depth analysis discussing prompt-specific behavior of different language models implemented.}, number={13}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Singh, Shubhankar and Pupneja, Anirudh and Mital, Shivaansh and Shah, Cheril and Bawkar, Manish and Gupta, Lakshman Prasad and Kumar, Ajit and Kumar, Yaman and Gupta, Rushali and Ratn Shah, Rajiv}, year={2024}, month={Jul.}, pages={15955-15963} }