AIConfPaper
← Search

Zahra Ashktorab

4 accepted papers

2026

Wavelength.AI: Extending the Collaborative Game Wavelength as a Testbed for Studying Shared Understanding in Human–Agent Collaboration

Katelyn Morrison, Gabriel Gonzalez, Zahra Ashktorab, Matt Riemer, Andrew Anderson, Djallel Bouneffouf +1
IJCAI 2026

AI's increasing role as a personal agent assisting knowledge workers in everyday tasks underscores the need to investigate how to help human–agent teams build a shared understanding. We extend the collaborative "mind-reading" game Wavelength to include an AI teammate, presenting the first demonstrat

Cited by 0Scholar ↗
2025

EvalAssist: LLM-as-a-Judge Simplified

Michael Desmond, Zahra Ashktorab, Werner Geyer, Elizabeth M. Daly, Martín Santillán Cooper, Qian Pan +3
AAAI 2025technical

We present EvalAssist, a framework that simplifies the LLM- as-a-judge workflow. The system provides an online criteria development environment, where users can interactively build, test, and share custom evaluation criteria in a structured and portable format. A library of LLM based evaluators is m…

Cited by 1Source ↗PDF ↗Scholar ↗
2025

Position: Theory of Mind Benchmarks are Broken for Large Language Models

Matthew Riemer, Zahra Ashktorab, Djallel Bouneffouf, Payel Das, Miao Liu, Justin D. Weisz +1
ICML 2025poster

Our paper argues that the majority of theory of mind benchmarks are broken because of their inability to directly test how large language models (LLMs) adapt to new partners. This problem stems from the fact that theory of mind benchmarks for LLMs are overwhelmingly inspired by the methods used to t…

Cited by 0Source ↗PDF ↗Scholar ↗
2021

Mental Models of AI Agents in a Cooperative Game Setting (Extended Abstract)

Katy Ilonka Gero, Zahra Ashktorab, Casey Dugan, Qian Pan, James Johnson, Werner Geyer +6
IJCAI 2021poster

As more and more forms of AI become prevalent, it becomes increasingly important to understand how people develop mental models of these systems. In this work we study people's mental models of an AI agent in a cooperative word guessing game. We run a study in which people play the game with an AI a…

Cited by 0Source ↗PDF ↗Scholar ↗