MarioNette: Self-Supervised Sprite Learning
Dmitriy Smirnov, MICHAEL GHARBI, Matthew Fisher, Vitor Campagnolo Guizilini, Alexei A Efros, Justin Solomon
Abstract
Artists and video game designers often construct 2D animations using libraries of sprites---textured patches of objects and characters. We propose a deep learning approach that decomposes sprite-based video animations into a disentangled representation of recurring graphic elements in a self-supervised manner. By jointly learning a dictionary of possibly transparent patches and training a network that places them onto a canvas, we deconstruct sprite-based content into a sparse, consistent, and explicit representation that can be easily used in downstream tasks, like editing or analysis. Our framework offers a promising approach for discovering recurring visual patterns in image collections without supervision.
BibTeX
@inproceedings{
smirnov2021marionette,
title={MarioNette: Self-Supervised Sprite Learning},
author={Dmitriy Smirnov and MICHAEL GHARBI and Matthew Fisher and Vitor Campagnolo Guizilini and Alexei A Efros and Justin Solomon},
booktitle={Advances in Neural Information Processing Systems},
editor={A. Beygelzimer and Y. Dauphin and P. Liang and J. Wortman Vaughan},
year={2021},
url={https://openreview.net/forum?id=3zP6RrQtNa}
}