Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentation
Zhiyuan Liang, Tiancai Wang, Xiangyu Zhang, Jian Sun, Jianbing Shen
Abstract
Sparsely annotated semantic segmentation (SASS) aims to train a segmentation network with coarse-grained (i.e.,point-, scribble-, and block-wise) supervisions, where only a small proportion of pixels are labeled in each image. In this paper, we propose a novel tree energy loss for SASS by providing semantic guidance for unlabeled pixels. The tree energy loss represents images as minimum spanning trees to model both low-level and high-level pair-wise affinities. By sequentially applying these affinities to the network prediction, soft pseudo labels for unlabeled pixels are generated in a coarse-to-fine manner, resulting in dynamic online self-training. The tree energy loss is effective and easy to be incorporated into existing frameworks by combining it with a traditional segmentation loss. Compared with previous SASS methods, our method requires no multi-stage training strategies, alternating optimization procedures, additional supervised data, or time-consuming post-processing while outperforming them in all types of supervised settings. Code is available at https://github.com/megvii-research/TreeEnergyLoss.
BibTeX
@inproceedings{cvpr2022_treeenergylossto,
title = {Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentation},
author = {Zhiyuan Liang and Tiancai Wang and Xiangyu Zhang and Jian Sun and Jianbing Shen},
booktitle = {CVPR 2022},
year = {2022}
}