Scaled and square-root elastic net

Abstract

In scaled lasso, the unknown regression coefficients and the scale parameter of the error distribution are estimated jointly. In lasso, the optimal penalty parameter is well-known to depend on the error scale, and it is therefore typically chosen using cross-validation. The main benefit of scaled lasso is that the penalty parameter is scale-free and can be predetermined from pure theoretical considerations. Nevertheless, scaled lasso performs poorly when there exist strong correlations between the predictors. As a remedy, we propose two different scaled elastic net (EN) formulations and derive convergent algorithms for their computation. The first formulation uses a conventional EN penalty whereas the second formulation differs from the former in that the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -loss is not squared. The former approach is referred to as the scaled EN estimator and the latter as the square-root EN estimator. We illustrate via numerical examples and simulations that the proposed methods outperform the scaled lasso, especially in the presence of high mutual coherence in the feature space.