Wormhole Learning

Abstract

Typically, to enlarge the operating domain of an object detector, more labeled training data is required. We describe a method called wormhole learning, which allows to extend the operating domain without additional data, but only with temporary access to an auxiliary sensor with certain invariance properties. We describe the instantiation of this principle with a regular visible-light RGB camera as the main sensor, and an infrared sensor as the temporary sensor. We start with a pre-trained RGB detector; then we train the infrared detector based on the RGB-inferred labels; finally we re-train the RGB detector based on the infrared-inferred labels. After these two transfer-learning steps, the RGB detector has enlarged its operating domain by inheriting part of the invariance to illumination of the infrared sensor; in particular, the RGB detector is now able to see much better at night. We analyze the wormhole learning phenomenon by bounding the possible gain in accuracy using mutual information properties of the two sensors and considered operating domain.