Piecewise linear regularized solution paths

We consider the generic regularized optimization problem $\hat{\mathsf{\beta}}(\lambda)=\arg \min_{\beta}L({\sf{y}},X{\sf{\beta}})+\lambda J({\sf{\beta}})$. Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407--499] have shown that for the LASSO--that is, if $L$ is squared error loss and $J(\beta)=\|\beta\|_1$ is the $\ell_1$ norm of $\beta$--the optimal coefficient path is piecewise linear, that is, $\partial \hat{\beta}(\lambda)/\partial \lambda$ is piecewise constant. We derive a general characterization of the properties of (loss $L$, penalty $J$) pairs which give piecewise linear coefficient paths. Such pairs allow for efficient generation of the full regularized coefficient paths. We investigate the nature of efficient path following algorithms which arise. We use our results to suggest robust versions of the LASSO for regression and classification, and to develop new, efficient algorithms for existing problems in the literature, including Mammen and van de Geer's locally adaptive regression splines.Comment: Published at http://dx.doi.org/10.1214/009053606000001370 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Classifying 50 years of Bell inequalities

Since John S. Bell demonstrated the interest of studying linear combinations of probabilities in relation with the EPR paradox in 1964, Bell inequalities have lead to numerous developments. Unfortunately, the description of Bell inequalities is subject to several degeneracies, which make any exchange of information about them unnecessarily hard. Here, we analyze these degeneracies and propose a decomposition for Bell-like inequalities based on a set of reference expressions which is not affected by them. These reference expressions set a common ground for comparing Bell inequalities. We provide algorithms based on finite group theory to compute this decomposition. Implementing these algorithms allows us to set up a compendium of reference Bell-like inequalities, available online at http://www.faacets.com . This website constitutes a platform where registered Bell-like inequalities can be explored, new inequalities can be compared to previously-known ones and relevant information on Bell inequalities can be added in a collaborative manner.Comment: 13+10 pages, 10 figures, submitted to the special edition of JPA for the 50 years of Bell's theorem. Comments especially welcome. See also faacets.co