Improved Small Sample Hypothesis Testing Using the Uncertain Likelihood Ratio

Abstract

This paper studies the problem of event detection in highly dynamic and uncertain environments where training data is limited and the number of observations is small. The objective is to determine if the training data and observations are drawn from the same or different distributions. The traditional approach is to use the Generalized Likelihood Ratio (GLR) test statistic. However, with limited training data and observations, the GLR is suboptimal. To overcome this, we propose to use the Uncertain Likelihood Ratio (ULR) test statistic to accurately account for the uncertainty. We show that the ULR is the mean most powerful test and that it is asymptotically equivalent to the GLR under a Jeffreys Prior. Furthermore, a simulation study validates the theory that the performance of the ULR is superior to the GLR when testing in the small sample regime.