Breaking Inter-Layer Co-Adaptation by Classifier Anonymization
Ikuro Sato, Kohta Ishikawa, Guoqing Liu, Masayuki Tanaka
Abstract
This study addresses an issue of co-adaptation between a feature extractor and a classifier in a neural network. A naive joint optimization of a feature extractor and a classifier often brings situations in which an excessively complex feature distribution adapted to a very specific classifier degrades the test performance. We introduce a method called Feature-extractor Optimization through Classifier Anonymization (FOCA), which is designed to avoid an explicit co-adaptation between a feature extractor and a particular classifier by using many randomly-generated, weak classifiers during optimization. We put forth a mathematical proposition that states the FOCA features form a point-like distribution within the same class in a class-separable fashion under special conditions. Real-data experiments under more general conditions provide supportive evidences.
BibTeX
@InProceedings{pmlr-v97-sato19a,
title = {Breaking Inter-Layer Co-Adaptation by Classifier Anonymization},
author = {Sato, Ikuro and Ishikawa, Kohta and Liu, Guoqing and Tanaka, Masayuki},
booktitle = {Proceedings of the 36th International Conference on Machine Learning},
pages = {5619--5627},
year = {2019},
editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan},
volume = {97},
series = {Proceedings of Machine Learning Research},
month = {09--15 Jun},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v97/sato19a/sato19a.pdf},
url = {https://proceedings.mlr.press/v97/sato19a.html},
abstract = {This study addresses an issue of co-adaptation between a feature extractor and a classifier in a neural network. A naive joint optimization of a feature extractor and a classifier often brings situations in which an excessively complex feature distribution adapted to a very specific classifier degrades the test performance. We introduce a method called Feature-extractor Optimization through Classifier Anonymization (FOCA), which is designed to avoid an explicit co-adaptation between a feature extractor and a particular classifier by using many randomly-generated, weak classifiers during optimization. We put forth a mathematical proposition that states the FOCA features form a point-like distribution within the same class in a class-separable fashion under special conditions. Real-data experiments under more general conditions provide supportive evidences.}
}