Explainable Action Prediction through Self-Supervision on Scene Graphs

Pawit Kochakarn; Daniele De Martini; Daniel Omeiza; Lars Kunze

ICRA 2023poster13 citations

Explainable Action Prediction through Self-Supervision on Scene Graphs

Pawit Kochakarn, Daniele De Martini, Daniel Omeiza, Lars Kunze

Abstract

This work explores scene graphs as a distilled representation of high-level information for autonomous driving, applied to future driver-action prediction. Given the scarcity and strong imbalance of data samples, we propose a self-supervision pipeline to infer representative and well-separated embeddings. Key aspects are interpretability and explainability; as such, we embed in our architecture attention mechanisms that can create spatial and temporal heatmaps on the scene graphs. We evaluate our system on the ROAD dataset against a fully-supervised approach, showing the superiority of our training regime.

BibTeX

@inproceedings{icra2023_explainableactio,
  title = {Explainable Action Prediction through Self-Supervision on Scene Graphs},
  author = {Pawit Kochakarn and Daniele De Martini and Daniel Omeiza and Lars Kunze},
  booktitle = {ICRA 2023},
  year = {2023}
}