Coloured Petri Nets for Monitoring Human Actions in Flexible Human-Robot Teams

Abstract

Human-robot collaboration in shared workspaces enables companies to improve efficiency and the quality of work for human workers. A novel research direction in this field is that human and robot dynamically negotiate which actions to perform. This requires the robot to permanently monitor the current task state and actions the human has performed. We envision a system that tracks the task progress based on visual input. We assume that a task specification is provided in terms of a coloured Petri net. The contribution of this paper is twofold. We introduce the notion of emissions for coloured Petri nets that incorporate partial observability and uncertainty. We then show how one can efficiently determine the evolution of the net when a sequence of partial observations is provided. To this end, we determine a small set of transition as candidates in a firing sequence first. Then, transition sequences are sampled to obtain markings compatible with the latest observation. We evaluate our algorithm in a simulated environment on a pick and place task and compare it to an aging-based approach from literature. Results show that our algorithm achieves higher precision compared to the aging-based approach. Running times indicate that the update procedure can run at 15 Hz on average.