Wavelet block thresholding for samples with random design: a minimax approach under the LpL^p risk

We consider the regression model with (known) random design. We investigate the minimax performances of an adaptive wavelet block thresholding estimator under the $\mathbb{L}^p$ risk with $p\ge 2$ over Besov balls. We prove that it is near optimal and that it achieves better rates of convergence than the conventional term-by-term estimators (hard, soft,...).Comment: Published at http://dx.doi.org/10.1214/07-EJS067 in the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org

Adapting to Unknown Smoothness by Aggregation of Thresholded Wavelet Estimators

We study the performances of an adaptive procedure based on a convex combination, with data-driven weights, of term-by-term thresholded wavelet estimators. For the bounded regression model, with random uniform design, and the nonparametric density model, we show that the resulting estimator is optimal in the minimax sense over all Besov balls under the $L^2$ risk, without any logarithm factor

Wavelet Estimation Via Block Thresholding : A Minimax Study Under The LpL^p Risk

19 pagesWe investigate the asymptotic minimax properties of an adaptive wavelet block thresholding estimator under the ${L}^p$ risk over Besov balls. It can be viewed as a $\mathbb{L}^p$ version of the BlockShrink estimator developed by Cai (1996,1997,2002). Firstly, we show that it is (near) optimal for numerous statistical models, including certain inverse problems. Under this statistical context, it achieves better rates of convergence than the hard thresholding estimator introduced by Donoho and Johnstone (1995). Secondly, we apply this general result to a deconvolution problem

Numerical performances of a warped wavelet estimation procedure for regression in random design

The purpose of this paper is to investigate the numerical performances of the hard thresholding procedure introduced by Kerkyacharian and Picard (2004) for the non-parametric regression model with random design. That construction adopts a new approach by using a wavelet basis warped with a function depending on the design, which enables to estimate regression functions under mild assumptions on the design. We compare our numerical properties to those obtained for other constructions based on hard wavelet thresholding. The performances are evaluated on numerous simulated data sets covering a broad variety of settings including known and unknown design density models, and also on real data sets

A maxiset approach of a Gaussian noise model

We consider the problem of estimating an unknown function $f$ in a homoscedastic Gaussian white noise setting under $\mathbb{L}^p$ risk. The particularity of this model is that it has an intermediate function, say $v$, which complicates the estimate significantly. While varying the assumptions on $v$, we investigate the minimax rate of convergence over two balls of spaces which belong to family of Besov classes. One is defined as usual and the other called 'weighted Besov balls' used $v$ explicitly. Adopting the maxiset approach, we develop a natural hard thresholding procedure which attained the minimax rate of convergence within a logarithmic factor over these weighted balls

A maxiset approach of a Gaussian white noise model

This paper is devoted to the estimation of an unknown function $f$ in the framework of a Gaussian white noise model. The noise process is represented by $t\rightarrow\frac{1}{\sqrt{n}}\int_{0}^{t}g(x) dB_x$, where the variance function $g$ is assumed to be known. Adopting the maxiset point of view, we study the performance of two different hard thresholding estimators in $\mathbb{L}^p$ norm. In a first part, we expand $f$ on a compactly supported wavelet basis $\{\psi_{\lambda}(.); \ \lambda\in\Lambda\}$. From this decomposition, we use some results about the heteroscedastic white noise model to construct a well adapted hard thresholding estimator and to exhibit the associated maxiset. In a second part, we introduce the classes of Muckenhoupt weights and we use this analytical tools to investigate the geometrical properties of warped wavelet basis $\{\psi_{\lambda}(T(.)); \ \lambda\in\Lambda\}$ in $\mathbb{L}^p$ norm. Expanding $f$ on such a basis and considering the associated hard thresholding estimator, we investigate the maxiset properties under some assumptions on $g$. We finally apply this result to find an upper bound over weighted Besov spaces

On adaptive wavelet estimation of a class of weighted densities

We investigate the estimation of a weighted density taking the form $g=w(F)f$, where $f$ denotes an unknown density, $F$ the associated distribution function and $w$ is a known (non-negative) weight. Such a class encompasses many examples, including those arising in order statistics or when $g$ is related to the maximum or the minimum of $N$ (random or fixed) independent and identically distributed (\iid) random variables. We here construct a new adaptive non-parametric estimator for $g$ based on a plug-in approach and the wavelets methodology. For a wide class of models, we prove that it attains fast rates of convergence under the $\mathbb{L}_p$ risk with $p\ge 1$ (not only for $p = 2$ corresponding to the mean integrated squared error) over Besov balls. The theoretical findings are illustrated through several simulations

Block thresholding for wavelet-based estimation of function derivatives from a heteroscedastic multichannel convolution model

We observe $n$ heteroscedastic stochastic processes $\{Y_v(t)\}_{v}$, where for any $v\in\{1,\ldots,n\}$ and $t \in [0,1]$, $Y_v(t)$ is the convolution product of an unknown function $f$ and a known blurring function $g_v$ corrupted by Gaussian noise. Under an ordinary smoothness assumption on $g_1,\ldots,g_n$, our goal is to estimate the $d$-th derivatives (in weak sense) of $f$ from the observations. We propose an adaptive estimator based on wavelet block thresholding, namely the "BlockJS estimator". Taking the mean integrated squared error (MISE), our main theoretical result investigates the minimax rates over Besov smoothness spaces, and shows that our block estimator can achieve the optimal minimax rate, or is at least nearly-minimax in the least favorable situation. We also report a comprehensive suite of numerical simulations to support our theoretical findings. The practical performance of our block estimator compares very favorably to existing methods of the literature on a large set of test functions

On the estimation of density-weighted average derivative by wavelet methods under various dependence structures

International audienceThe problem of estimating the density-weighted average derivative of a regression function is considered. We present a new consistent estimator based on a plug-in approach and wavelet projections. Its performances are explored under various dependence structures on the observations: the independent case, the $\rho$-mixing case and the $\alpha$-mixing case. More precisely, denoting $n$ the number of observations, in the independent case, we prove that it attains $1/n$ under the mean squared error, in the $\rho$-mixing case, $1/\sqrt{n}$ under the mean absolute error, and, in the $\alpha$-mixing case, $\sqrt{\ln n /n}$ under the mean absolute error. A short simulation study illustrates the theory

The degrees of freedom of the Lasso for general design matrix

In this paper, we investigate the degrees of freedom (\dof) of penalized $\ell_1$ minimization (also known as the Lasso) for linear regression models. We give a closed-form expression of the \dof of the Lasso response. Namely, we show that for any given Lasso regularization parameter $\lambda$ and any observed data $y$ belonging to a set of full (Lebesgue) measure, the cardinality of the support of a particular solution of the Lasso problem is an unbiased estimator of the degrees of freedom. This is achieved without the need of uniqueness of the Lasso solution. Thus, our result holds true for both the underdetermined and the overdetermined case, where the latter was originally studied in \cite{zou}. We also show, by providing a simple counterexample, that although the \dof theorem of \cite{zou} is correct, their proof contains a flaw since their divergence formula holds on a different set of a full measure than the one that they claim. An effective estimator of the number of degrees of freedom may have several applications including an objectively guided choice of the regularization parameter in the Lasso through the \sure framework. Our theoretical findings are illustrated through several numerical simulations.Comment: A short version appeared in SPARS'11, June 2011 Previously entitled "The degrees of freedom of penalized l1 minimization