Robust Parameter Estimation Based on the K-Divergence

Abstract

In this paper we present a new divergence, called $\mathcal{K}$-divergence, that involves a weighted version of the hypothesized log-likelihood function. To down-weight low density areas, attributed to outliers, the corresponding weight function is a convolved version of the underlying density with a strictly positive smoothing "$\mathcal{K}$"ernel function parameterized by a bandwidth parameter. The resulting minimum $\mathcal{K}$-divergence estimator $({\text{M}}\mathcal{K}{\text{DE}})$ operates by minimizing the empirical $\mathcal{K}$-divergence w.r.t. the vector parameter of interest. The ${\text{M}}\mathcal{K}{\text{DE}}$ utilizes Parzen's non-parametric kernel density estimator, arising from the nature of the weight function, to suppress outliers. By proper selection of the kernel's bandwidth parameter we show that the ${\text{M}}\mathcal{K}{\text{DE}}$ can gain enhanced estimation performance along with implementation simplicity as compared to other robust estimators.