Kevin Leach — accepted AI-conference papers

2024

De-Identification of Sensitive Personal Data in Datasets Derived from IIT-CDIP

Stefan Larson, Nicole Cornehl Lima, Santiago Pedroza Diaz, Amogh Manoj Joshi, Siddharth Betala, Jamiu Tunde Suleiman +6

EMNLP 2024main

The IIT-CDIP document collection is the source of several widely used and publicly accessible document understanding datasets. In this paper, manual inspection of 5 datasets derived from IIT-CDIP uncovers the presence of thousands of instances of sensitive personal data, including US Social Security…

Cited by 1 Source ↗PDF ↗code ↗Scholar ↗

2024

Generating Hard-Negative Out-of-Scope Data with ChatGPT for Intent Classification

Zhijian Li, Stefan Larson, Kevin Leach

COLING 2024main

Intent classifiers must be able to distinguish when a user’s utterance does not belong to any supported intent to avoid producing incorrect and unrelated system responses. Although out-of-scope (OOS) detection for intent classifiers has been studied, previous work has not yet studied changes in clas…

Cited by 1 Source ↗PDF ↗code ↗Scholar ↗

2022

Evaluating Out-of-Distribution Performance on Document Image Classifiers

Stefan Larson, Gordon Lim, Yutong Ai, David Kuang, Kevin Leach

NeurIPS 2022accept

The ability of a document classifier to handle inputs that are drawn from a distribution different from the training distribution is crucial for robust deployment and generalizability. The RVL-CDIP corpus is the de facto standard benchmark for document classification, yet to our knowledge all studie…

Cited by 22 Source ↗PDF ↗code ↗Scholar ↗

2022

One Agent To Rule Them All: Towards Multi-agent Conversational AI

Christopher Clarke, Joseph Peper, Karthik Krishnamurthy, Walter Talamonti, Kevin Leach, Walter Lasecki +3

ACL 2022findings

The increasing volume of commercially available conversational agents (CAs) on the market has resulted in users being burdened with learning and adopting multiple agents to accomplish their tasks. Though prior work has explored supporting a multitude of domains within the design of a single agent, t…

Cited by 19 Source ↗PDF ↗code ↗Scholar ↗

2020

Inconsistencies in Crowdsourced Slot-Filling Annotations: A Typology and Identification Methods

Stefan Larson, Adrian Cheung, Anish Mahendran, Kevin Leach, Jonathan K. Kummerfeld

COLING 2020main

Slot-filling models in task-driven dialog systems rely on carefully annotated training data. However, annotations by crowd workers are often inconsistent or contain errors. Simple solutions like manually checking annotations or having multiple workers label each sample are expensive and waste effort…

Cited by 16 Source ↗PDF ↗Scholar ↗