AIConfPaper
← Search

Mark K. Ho

11 accepted papers

2025

Estimating cognitive biases with attention-aware inverse planning

Sounak Banerjee, Daphne Cornelisse, Deepak Edakkattil Gopinath, Emily Sumner, Jonathan DeCastro, Guy Rosman +2
NeurIPS 2025spotlight

People's goal-directed behaviors are influenced by their cognitive biases, and autonomous systems that interact with people should be aware of this. For example, people's attention to objects in their environment will be biased in a way that systematically affects how they perform everyday tasks suc…

Cited by 0Source ↗Scholar ↗
2024

Structurally Guided Task Decomposition in Spatial Navigation Tasks (Student Abstract)

Ruiqi He, Carlos G. Correa, Tom L. Griffiths, Mark K. Ho
AAAI 2024technical

How are people able to plan so efficiently despite limited cognitive resources? We aimed to answer this question by extending an existing model of human task decomposition that can explain a wide range of simple planning problems by adding structure information to the task to facilitate planning in…

Cited by 1Source ↗PDF ↗Scholar ↗
2023

Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation

Andi Peng, Aviv Netanyahu, Mark K Ho, Tianmin Shu, Andreea Bobu, Julie Shah +1
ICML 2023poster

Policies often fail at test-time due to *distribution shifts*---changes in the state and reward that occur when an end user deploys the policy in environments different from those seen in training. Data augmentation can help models be more robust to such shifts by varying specific concepts in the st…

Cited by 18Source ↗PDF ↗Scholar ↗
2022

How to talk so AI will learn: Instructions, descriptions, and autonomy

Theodore Sumers, Robert D. Hawkins, Mark K Ho, Thomas L. Griffiths, Dylan Hadfield-Menell
NeurIPS 2022accept

From the earliest years of our lives, humans use language to express our beliefs and desires. Being able to talk to artificial agents about our preferences would thus fulfill a central goal of value alignment. Yet today, we lack computational models explaining such language use. To address this chal…

Cited by 25Source ↗PDF ↗code ↗Scholar ↗
2022

On the Expressivity of Markov Reward (Extended Abstract)

David Abel, Will Dabney, Anna Harutyunyan, Mark K. Ho, Michael L. Littman, Doina Precup +1
IJCAI 2022poster

Reward is the driving force for reinforcement-learning agents. We here set out to understand the expressivity of Markov reward as a way to capture tasks that we would want an agent to perform. We frame this study around three new abstract notions of "task": (1) a set of acceptable behaviors…

Cited by 0Source ↗PDF ↗Scholar ↗
2021

Learning Rewards From Linguistic Feedback

Theodore R. Sumers, Mark K. Ho, Robert D. Hawkins, Karthik Narasimhan, Thomas L. Griffiths
AAAI 2021technical

We explore unconstrained natural language feedback as a learning signal for artificial agents. Humans use rich and varied language to teach, yet most prior work on interactive learning from language assumes a particular form of input (e.g., commands). We propose a general framework which does not ma…

Cited by 58Source ↗PDF ↗code ↗Scholar ↗
2021

On the Expressivity of Markov Reward

David Abel, Will Dabney, Anna Harutyunyan, Mark K Ho, Michael Littman, Doina Precup +1
NeurIPS 2021oral

Reward is the driving force for reinforcement-learning agents. This paper is dedicated to understanding the expressivity of reward as a way to capture tasks that we would want an agent to perform. We frame this study around three new abstract notions of “task” that might be desirable: (1) a set of a…

Cited by 117Source ↗PDF ↗Scholar ↗
2019

On the Utility of Learning about Humans for Human-AI Coordination

Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel +1
NeurIPS 2019poster

While we would like agents that can coordinate with humans, current algorithms such as self-play and population-based training create agents that can coordinate with themselves. Agents that assume their partner to be optimal or similar to them can converge to coordination protocols that fail to unde…

Cited by 513Source ↗PDF ↗code ↗Scholar ↗
2018

Learning Task Specifications from Demonstrations

Marcell Vazquez-Chanlatte, Susmit Jha, Ashish Tiwari, Mark K Ho, Sanjit Seshia
NeurIPS 2018poster

Real-world applications often naturally decompose into several sub-tasks. In many settings (e.g., robotics) demonstrations provide a natural way to specify the sub-tasks. However, most methods for learning from demonstrations either do not provide guarantees that the artifacts learned for th…

Cited by 96Source ↗PDF ↗Scholar ↗
2017

Interactive Learning from Policy-Dependent Human Feedback

James MacGlashan, Mark K. Ho, Robert Loftin, Bei Peng, Guan Wang, David L. Roberts +2
ICML 2017poster

This paper investigates the problem of interactively learning behaviors communicated by a human teacher using positive and negative feedback. Much previous work on this problem has made the assumption that people provide feedback for decisions that is dependent on the behavior they are teaching and…

Cited by 387Source ↗PDF ↗Scholar ↗
2016

Showing versus doing: Teaching by demonstration

Mark K Ho, Michael Littman, James MacGlashan, Fiery Cushman, Joseph L Austerweil
NeurIPS 2016oral

People often learn from others' demonstrations, and classic inverse reinforcement learning (IRL) algorithms have brought us closer to realizing this capacity in machines. In contrast, teaching by demonstration has been less well studied computationally. Here, we develop a novel Bayesian model for te…

Cited by 143Source ↗PDF ↗Scholar ↗