Deterministic Feature Decoupling by Surfing Invariance Manifolds

Abstract

We introduce a formalism that justifies and extends a heuristic method for algebraically decoupling deterministic features that recently proved useful for improving feature-based classification. Our new formalism is based on defining transformations inside manifolds, by following trajectories along the features' gradients. Through these transformations we define a normalization that, we demonstrate, allows for decoupling differentiable features. By applying this to sampling moments, we obtain a quasi-analytic solution for the orthokurtosis, a modification of the kurtosis that is not just decoupled from mean and variance, but also from skewness. After theoretically motivating feature decoupling for random data distributions, we illustrate with a regression problem example how decoupled features may perform significantly better than coupled ones.