Unlocking Slot Attention by Changing Optimal Transport Costs
Yan Zhang, David W Zhang, Simon Lacoste-Julien, Gertjan J. Burghouts, Cees G. M. Snoek
Abstract
Slot attention is a powerful method for object-centric modeling in images and videos. However, its set-equivariance limits its ability to handle videos with a dynamic number of objects because it cannot break ties. To overcome this limitation, we first establish a connection between slot attention and optimal transport. Based on this new perspective we propose **MESH** (Minimize Entropy of Sinkhorn): a cross-attention module that combines the tiebreaking properties of unregularized optimal transport with the speed of regularized optimal transport. We evaluate slot attention using MESH on multiple object-centric learning benchmarks and find significant improvements over slot attention in every setting.
BibTeX
@inproceedings{icml2023_unlockingslotatt,
title = {Unlocking Slot Attention by Changing Optimal Transport Costs},
author = {Yan Zhang and David W Zhang and Simon Lacoste-Julien and Gertjan J. Burghouts and Cees G. M. Snoek},
booktitle = {ICML 2023},
year = {2023}
}