Kai-Siang Ang

2 accepted papers

2021

Refining Targeted Syntactic Evaluation of Language Models

Benjamin Newman, Kai-Siang Ang, Julia Gong, John Hewitt

NAACL 2021long

Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models’ syntactic knowledge using hand-crafted minimal pairs of sentences that differ only in the main verb’s conjugation. The method evaluates whether language models rate each grammatical sentence as…

Cited by 49 Source ↗PDF ↗code ↗Scholar ↗

2019

On the Accuracy of Influence Functions for Measuring Group Effects

Pang Wei W Koh, Kai-Siang Ang, Hubert Teo, Percy Liang

NeurIPS 2019poster

Influence functions estimate the effect of removing a training point on a model without the need to retrain. They are based on a first-order Taylor approximation that is guaranteed to be accurate for sufficiently small changes to the model, and so are commonly used to study the effect of individual…

Cited by 243 Source ↗PDF ↗code ↗Scholar ↗