A Probability Gradient Based Approach for Sampling Boundaries of In-Domain Data

Abstract

In machine learning applications, it is desirable to distinguish between in-domain and out-of-domain data. However, in most cases, only in-domain data is available and consequently identifying the ‘boundary’ between in-domain and out-of-domain is a significant challenge. In this paper we present a novel technique that can identify points on this boundary based on only in-domain data. Specifically, the proposed method operates on the hypothesis that the gradient of the probability of the data being in-domain will be highest at the boundary. It utilises an iterative approach, alternating between Monte Carlo sampling of an estimated ‘boundary distribution’ and a binary classifier trained on these points to distinguish between in-domain and out-of-domain data to improve the estimate of the boundary. This method leads to both a set of high-quality data points from the boundary and a calibrated out-of-domain detector. The operation of the proposed approach is validated on MNIST, Fashion-MNIST, and Omniglot datasets.