Efficient Smoothed Concomitant Lasso Estimation for High Dimensional Regression

Abstract : In high dimensional settings, sparse structures are crucial for efficiency, both in term of memory, computation and performance. It is customary to consider 1 penalty to enforce spar-sity in such scenarios. Sparsity enforcing methods, the Lasso being a canonical example, are popular candidates to address high dimension. For efficiency, they rely on tuning a parameter trading data fitting versus sparsity. For the Lasso theory to hold this tuning parameter should be proportional to the noise level, yet the latter is often unknown in practice. A possible remedy is to jointly optimize over the regression parameter as well as over the noise level. This has been considered under several names in the literature: Scaled-Lasso, Square-root Lasso, Concomitant Lasso estimation for instance, and could be of interest for confidence sets or uncertainty quantification. In this work, after illustrating numerical difficulties for the Smoothed Concomitant Lasso formulation, we propose a modification we coined Smoothed Concomitant Lasso, aimed at increasing numerical stability. We propose an efficient and accurate solver leading to a computational cost no more expansive than the one for the Lasso. We leverage on standard ingredients behind the success of fast Lasso solvers: a coordinate descent algorithm, combined with safe screening rules to achieve speed efficiency, by eliminating early irrelevant features.
Keywords : Lasso regression
Type de document :
Article dans une revue
J. Phys.: Conference Series , 2017, Journal of Physics: Conference Series, J. Phys.: Conf. Ser. 904 012006, 〈10.1088/1742-6596/904/1/012006〉
Liste complète des métadonnées

Littérature citée [28 références]  Voir  Masquer  Télécharger

https://hal-enpc.archives-ouvertes.fr/hal-01404966
Contributeur : Vincent Leclère <>
Soumis le : mardi 29 novembre 2016 - 13:49:32
Dernière modification le : jeudi 18 janvier 2018 - 11:02:50
Document(s) archivé(s) le : lundi 27 mars 2017 - 08:43:12

Fichier

1606.02702v1.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, Vincent Leclère, Joseph Salmon. Efficient Smoothed Concomitant Lasso Estimation for High Dimensional Regression. J. Phys.: Conference Series , 2017, Journal of Physics: Conference Series, J. Phys.: Conf. Ser. 904 012006, 〈10.1088/1742-6596/904/1/012006〉. 〈hal-01404966〉

Partager

Métriques

Consultations de la notice

285

Téléchargements de fichiers

129