Learning Heteroscedastic Models by Convex Programming under Group Sparsity

Arnak S. Dalalyan (1) , Mohamed Hebiri (2) , Katia Meziani (3) , Joseph Salmon (4)

1 CREST - Centre de Recherche en Économie et Statistique
2 LAMA - Laboratoire d'Analyse et de Mathématiques Appliquées
3 CEREMADE - CEntre de REcherches en MAthématiques de la DEcision
4 TSI - Département Traitement du Signal et des Images

Popular sparse estimation methods based on $\ell_1$-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutes a major obstacle in applying these methods in several frameworks---such as time series, random fields, inverse problems---for which the noise is rarely homoscedastic and its level is hard to know in advance. In this paper, we propose a new approach to the joint estimation of the conditional mean and the conditional variance in a high-dimensional (auto-) regression setting. An attractive feature of the proposed estimator is that it is efficiently computable even for very large scale problems by solving a second-order cone program (SOCP). We present theoretical analysis and numerical results assessing the performance of the proposed procedure.

Keywords: Heteroscedastic regression; Group Sparsity; Time Series Prediction

Domains: Machine Learning [stat.ML]; Statistics Theory [stat.TH]

Arnak S. Dalalyan, Mohamed Hebiri, Katia Meziani, Joseph Salmon. Learning Heteroscedastic Models by Convex Programming under Group Sparsity. Proceedings of the 30 th International Conference on Machine Learning, 2013, http://icml.cc/2013/?page_id=43. ⟨hal-00813908⟩