Chiranthan Sridhar

MALAMUTE: A Multilingual, Highly-granular, Template-free, Education-based Probing Dataset

Language models (LMs) have excelled in various broad domains. However, to ensure their safe and effective integration into real-world educational settings, they must demonstrate proficiency in specific, granular areas of knowledge. Existing cloze-style benchmarks, commonly used to evaluate LMs’ know…