Qian Pan — accepted AI-conference papers

2025

EvalAssist: LLM-as-a-Judge Simplified

Michael Desmond, Zahra Ashktorab, Werner Geyer, Elizabeth M. Daly, Martín Santillán Cooper, Qian Pan +3

AAAI 2025technical

We present EvalAssist, a framework that simplifies the LLM- as-a-judge workflow. The system provides an online criteria development environment, where users can interactively build, test, and share custom evaluation criteria in a structured and portable format. A library of LLM based evaluators is m…

Cited by 1 Source ↗PDF ↗Scholar ↗

2025

Granite Guardian: Comprehensive LLM Safeguarding

Inkit Padhi, Manish Nagireddy, Giandomenico Cornacchia, Subhajit Chaudhury, Tejaswini Pedapati, Pierre Dognin +16

NAACL 2025industry

The deployment of language models in real-world applications exposes users to various risks, including hallucinations and harmful or unethical content. These challenges highlight the urgent need for robust safeguards to ensure safe and responsible AI. To address this, we introduce Granite Guardian,…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗

2021

Mental Models of AI Agents in a Cooperative Game Setting (Extended Abstract)

Katy Ilonka Gero, Zahra Ashktorab, Casey Dugan, Qian Pan, James Johnson, Werner Geyer +6

IJCAI 2021poster

As more and more forms of AI become prevalent, it becomes increasingly important to understand how people develop mental models of these systems. In this work we study people's mental models of an AI agent in a cooperative word guessing game. We run a study in which people play the game with an AI a…

Cited by 0 Source ↗PDF ↗Scholar ↗