Variance Reduced Stochastic Gradient Descent with Neighbors

Stochastic Gradient Descent (SGD) is a workhorse in machine learning, yet its slow convergence can be a computational bottleneck. Variance reduction techniques such as SAG, SVRG and SAGA have been proposed to overcome this weakness, achieving linear convergence. However, these methods are either based on computations of full gradients at pivot points, or on keeping per data point corrections in memory. Therefore speed-ups relative to SGD may need a minimal number of epochs in order to materialize. This paper investigates algorithms that can exploit neighborhood structure in the training data to share and re-use information about past stochastic gradients across data points, which offers advantages in the transient optimization phase. As a side-product we provide a unified convergence analysis for a family of variance reduction algorithms, which we call memorization algorithms. We provide experimental results supporting our theory.Comment: Appears in: Advances in Neural Information Processing Systems 28 (NIPS 2015). 13 page

Poncelet's Theorem, Paraorthogonal Polynomials and the Numerical Range of Compressed Multiplication Operators

There has been considerable recent literature connecting Poncelet's theorem to ellipses, Blaschke products and numerical ranges, summarized, for example, in the recent book [11]. We show how those results can be understood using ideas from the theory of orthogonal polynomials on the unit circle (OPUC) and, in turn, can provide new insights to the theory of OPUC.Comment: 46 pages, 4 figures; minor revisions from v1; accepted for publication in Adv. Mat

Accelerating vaccine development and deployment: report of a Royal Society satellite meeting.

The Royal Society convened a meeting on the 17th and 18th November 2010 to review the current ways in which vaccines are developed and deployed, and to make recommendations as to how each of these processes might be accelerated. The meeting brought together academics, industry representatives, research sponsors, regulators, government advisors and representatives of international public health agencies from a broad geographical background. Discussions were held under Chatham House rules. High-throughput screening of new vaccine antigens and candidates was seen as a driving force for vaccine discovery. Multi-stakeholder, small-scale manufacturing facilities capable of rapid production of clinical grade vaccines are currently too few and need to be expanded. In both the human and veterinary areas, there is a need for tiered regulatory standards, differentially tailored for experimental and commercial vaccines, to allow accelerated vaccine efficacy testing. Improved cross-fertilization of knowledge between industry and academia, and between human and veterinary vaccine developers, could lead to more rapid application of promising approaches and technologies to new product development. Identification of best-practices and development of checklists for product development plans and implementation programmes were seen as low-cost opportunities to shorten the timeline for vaccine progression from the laboratory bench to the people who need it

Softening and Yielding of Soft Glassy Materials

Solids deform and fluids flow, but soft glassy materials, such as emulsions, foams, suspensions, and pastes, exhibit an intricate mix of solid and liquid-like behavior. While much progress has been made to understand their elastic (small strain) and flow (infinite strain) properties, such understanding is lacking for the softening and yielding phenomena that connect these asymptotic regimes. Here we present a comprehensive framework for softening and yielding of soft glassy materials, based on extensive numerical simulations of oscillatory rheological tests, and show that two distinct scenarios unfold depending on the material's packing density. For dense systems, there is a single, pressure-independent strain where the elastic modulus drops and the particle motion becomes diffusive. In contrast, for weakly jammed systems, a two-step process arises: at an intermediate softening strain, the elastic and loss moduli both drop down and then reach a new plateau value, whereas the particle motion becomes diffusive at the distinctly larger yield strain. We show that softening is associated with an extensive number of microscopic contact changes leading to a non-analytic rheological signature. Moreover, the scaling of the softening strain with pressure suggest the existence of a novel pressure scale above which softening and yielding coincide, and we verify the existence of this crossover scale numerically. Our findings thus evidence the existence of two distinct classes of soft glassy materials -- jamming dominated and dense -- and show how these can be distinguished by their rheological fingerprint.Comment: 9 pages, 11 figures, to appear in Soft Matte

Soft Sphere Packings at Finite Pressure but Unstable to Shear

When are athermal soft sphere packings jammed ? Any experimentally relevant definition must at the very least require a jammed packing to resist shear. We demonstrate that widely used (numerical) protocols in which particles are compressed together, can and do produce packings which are unstable to shear - and that the probability of generating such packings reaches one near jamming. We introduce a new protocol that, by allowing the system to explore different box shapes as it equilibrates, generates truly jammed packings with strictly positive shear moduli G. For these packings, the scaling of the average of G is consistent with earlier results, while the probability distribution P(G) exhibits novel and rich scalingComment: 5 pages, 6 figures. Resubmitted to Physical Review Letters after a few change

Aspirin desensitisation therapy for aspirin-intolerant chronic rhinosinusitis

This review is published as a Cochrane Review in the Cochrane Database of Systematic Reviews 2009, Issue 4. Cochrane Reviews are regularly updated as new evidence emerges and in response to comments and criticisms, and the Cochrane Database of Systematic Reviews should be consulted for the most recent version of the Review.</p

Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks

We present a technique for efficiently synthesizing images of atmospheric clouds using a combination of Monte Carlo integration and neural networks. The intricacies of Lorenz-Mie scattering and the high albedo of cloud-forming aerosols make rendering of clouds---e.g. the characteristic silverlining and the "whiteness" of the inner body---challenging for methods based solely on Monte Carlo integration or diffusion theory. We approach the problem differently. Instead of simulating all light transport during rendering, we pre-learn the spatial and directional distribution of radiant flux from tens of cloud exemplars. To render a new scene, we sample visible points of the cloud and, for each, extract a hierarchical 3D descriptor of the cloud geometry with respect to the shading location and the light source. The descriptor is input to a deep neural network that predicts the radiance function for each shading configuration. We make the key observation that progressively feeding the hierarchical descriptor into the network enhances the network's ability to learn faster and predict with high accuracy while using few coefficients. We also employ a block design with residual connections to further improve performance. A GPU implementation of our method synthesizes images of clouds that are nearly indistinguishable from the reference solution within seconds interactively. Our method thus represents a viable solution for applications such as cloud design and, thanks to its temporal stability, also for high-quality production of animated content.Comment: ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2017