← Search

Anjana Arunkumar

2 accepted papers

2022

Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks

EMNLP 2022main

How well can NLP models generalize to a variety of unseen tasks when provided with task instructions? To address this question, we first introduce Super-NaturalInstructions, a benchmark of 1,616 diverse NLP tasks and their expert-written instructions. Our collection covers 76 distinct task types, in…

2021

How Robust are Model Rankings : A Leaderboard Customization Approach for Equitable Evaluation

AAAI 2021technical

Models that top leaderboards often perform unsatisfactorily when deployed in real world applications; this has necessitated rigorous and expensive pre-deployment model testing. A hitherto unexplored facet of model performance is: Are our leaderboards doing equitable evaluation? In this paper, we int…