Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

This paper demonstrates theoretically and empirically that a greedy algorithm called Orthogonal Matching Pursuit (OMP) can reliably recover a signal with $m$ nonzero entries in dimension $d$ given ${rm O}(m ln d)$ random linear measurements of that signal. This is a massive improvement over previous results, which require ${rm O}(m^{2})$ measurements. The new results for OMP are comparable with recent results for another approach called Basis Pursuit (BP). In some settings, the OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal recovery problems

Simultaneous sparse approximation via greedy pursuit

A simple sparse approximation problem requests an approximation of a given input signal as a linear combination of T elementary signals drawn from a large, linearly dependent collection. An important generalization is simultaneous sparse approximation. Now one must approximate several input signals at once using different linear combinations of the same T elementary signals. This formulation appears, for example, when analyzing multiple observations of a sparse signal that have been contaminated with noise. A new approach to this problem is presented here: a greedy pursuit algorithm called simultaneous orthogonal matching pursuit. The paper proves that the algorithm calculates simultaneous approximations whose error is within a constant factor of the optimal simultaneous approximation error. This result requires that the collection of elementary signals be weakly correlated, a property that is also known as incoherence. Numerical experiments demonstrate that the algorithm often succeeds, even when the inputs do not meet the hypotheses of the proof

Applications of sparse approximation in communications

Sparse approximation problems abound in many scientific, mathematical, and engineering applications. These problems are defined by two competing notions: we approximate a signal vector as a linear combination of elementary atoms and we require that the approximation be both as accurate and as concise as possible. We introduce two natural and direct applications of these problems and algorithmic solutions in communications. We do so by constructing enhanced codebooks from base codebooks. We show that we can decode these enhanced codebooks in the presence of Gaussian noise. For MIMO wireless communication channels, we construct simultaneous sparse approximation problems and demonstrate that our algorithms can both decode the transmitted signals and estimate the channel parameters

Algorithmic linear dimension reduction in the l_1 norm for sparse vectors

This paper develops a new method for recovering m-sparse signals that is simultaneously uniform and quick. We present a reconstruction algorithm whose run time, O(m log^2(m) log^2(d)), is sublinear in the length d of the signal. The reconstruction error is within a logarithmic factor (in m) of the optimal m-term approximation error in l_1. In particular, the algorithm recovers m-sparse signals perfectly and noisy signals are recovered with polylogarithmic distortion. Our algorithm makes O(m log^2 (d)) measurements, which is within a logarithmic factor of optimal. We also present a small-space implementation of the algorithm. These sketching techniques and the corresponding reconstruction algorithms provide an algorithmic dimension reduction in the l_1 norm. In particular, vectors of support m in dimension d can be linearly embedded into O(m log^2 d) dimensions with polylogarithmic distortion. We can reconstruct a vector from its low-dimensional sketch in time O(m log^2(m) log^2(d)). Furthermore, this reconstruction is stable and robust under small perturbations

Improved sparse approximation over quasi-incoherent dictionaries

This paper discusses a new greedy algorithm for solving the sparse approximation problem over quasi-incoherent dictionaries. These dictionaries consist of waveforms that are uncorrelated "on average," and they provide a natural generalization of incoherent dictionaries. The algorithm provides strong guarantees on the quality of the approximations it produces, unlike most other methods for sparse approximation. Moreover, very efficient implementations are possible via approximate nearest-neighbor data structure

Sparse Approximation Via Iterative Thresholding

The well-known shrinkage technique is still relevant for contemporary signal processing problems over redundant dictionaries. We present theoretical and empirical analyses for two iterative algorithms for sparse approximation that use shrinkage. The GENERAL IT algorithm amounts to a Landweber iteration with nonlinear shrinkage at each iteration step. The BLOCK IT algorithm arises in morphological components analysis. A sufficient condition for which General IT exactly recovers a sparse signal is presented, in which the cumulative coherence function naturally arises. This analysis extends previous results concerning the Orthogonal Matching Pursuit (OMP) and Basis Pursuit (BP) algorithms to IT algorithms

Differential behavioral state-dependence in the burst properties of CA3 and CA1 neurons

Brief bursts of fast high-frequency action potentials are a signature characteristic of CA3 and CA1 pyramidal neurons. Understanding the factors determining burst and single spiking is potentially significant for sensory representation, synaptic plasticity and epileptogenesis. A variety of models suggest distinct functional roles for burst discharge, and for specific characteristics of the burst in neural coding. However, little in vivo data demonstrate how often and under what conditions CA3 and CA1 actually exhibit burst and single spike discharges. The present study examined burst discharge and single spiking of CA3 and CA1 neurons across distinct behavioral states (awake-immobility and maze-running) in rats. In both CA3 and CA1 spike bursts accounted for less than 20% of all spike events. CA3 neurons exhibited more spikes per burst, greater spike frequency, larger amplitude spikes and more spike amplitude attenuation than CA1 neurons. A major finding of the present study is that the propensity of CA1 neurons to burst was affected by behavioral state, while the propensity of CA3 to burst was not. CA1 neurons exhibited fewer bursts during maze running compared with awake-immobility. In contrast, there were no differences in burst discharge of CA3 neurons. Neurons in both subregions exhibited smaller spike amplitude, fewer spikes per burst, longer inter-spike intervals and greater spike amplitude attenuation within a burst during awake-immobility compared with maze running. These findings demonstrate that the CA1 network is under greater behavioral state-dependent regulation than CA3. The present findings should inform both theoretic and computational models of CA3 and CA1 function. © 2006 IBRO

User-friendly tail bounds for sums of random matrices

This paper presents new probability inequalities for sums of independent, random, self-adjoint matrices. These results place simple and easily verifiable hypotheses on the summands, and they deliver strong conclusions about the large-deviation behavior of the maximum eigenvalue of the sum. Tail bounds for the norm of a sum of random rectangular matrices follow as an immediate corollary. The proof techniques also yield some information about matrix-valued martingales. In other words, this paper provides noncommutative generalizations of the classical bounds associated with the names Azuma, Bennett, Bernstein, Chernoff, Hoeffding, and McDiarmid. The matrix inequalities promise the same diversity of application, ease of use, and strength of conclusion that have made the scalar inequalities so valuable.Comment: Current paper is the version of record. The material on Freedman's inequality has been moved to a separate note; other martingale bounds are described in Caltech ACM Report 2011-0

On the shopfloor: exploring the impact of teacher trade unions on school-based industrial relations

Teachers are highly unionised workers and their trade unions exert an important influence on the shaping and implementation of educational policy. Despite this importance there is relatively little analysis of the impact of teacher trade unions in educational management literature. Very little empirical research has sought to establish the impact of teacher unions at school level. In an era of devolved management and quasi-markets this omission is significant. New personnel issues continue to emerge at school level and this may well generate increased trade union activity at the workplace. This article explores the extent to which devolved management is drawing school-based union representation into a more prominent role. It argues that whilst there can be significant differences between individual schools, increased school autonomy is raising the profile of trade union activity in the workplace, and this needs to be better reflected in educational management research

Necessary and sufficient conditions of solution uniqueness in ℓ1\ell_1 minimization

This paper shows that the solutions to various convex $\ell_1$ minimization problems are \emph{unique} if and only if a common set of conditions are satisfied. This result applies broadly to the basis pursuit model, basis pursuit denoising model, Lasso model, as well as other $\ell_1$ models that either minimize $f(Ax-b)$ or impose the constraint $f(Ax-b)\leq\sigma$, where $f$ is a strictly convex function. For these models, this paper proves that, given a solution $x^*$ and defining I=\supp(x^*) and s=\sign(x^*_I), $x^*$ is the unique solution if and only if $A_I$ has full column rank and there exists $y$ such that $A_I^Ty=s$ and $|a_i^Ty|_\infty<1$ for $i\not\in I$. This condition is previously known to be sufficient for the basis pursuit model to have a unique solution supported on $I$. Indeed, it is also necessary, and applies to a variety of other $\ell_1$ models. The paper also discusses ways to recognize unique solutions and verify the uniqueness conditions numerically.Comment: 6 pages; revised version; submitte