Aadit Jain

Mistake Attribution: Fine-Grained Mistake Understanding in Egocentric Videos

We introduce Mistake Attribution (MATT), a new task for fine-grained understanding of human mistakes in egocentric videos. While prior work detects whether a mistake occurs, MATT attributes the mistake to what part of the instruction is violated (semantic role), when in the video the deviation becom