Size-biased permutation of a finite sequence with independent and identically distributed terms

This paper focuses on the size-biased permutation of $n$ independent and identically distributed (i.i.d.) positive random variables. This is a finite dimensional analogue of the size-biased permutation of ranked jumps of a subordinator studied in Perman-Pitman-Yor (PPY) [Probab. Theory Related Fields 92 (1992) 21-39], as well as a special form of induced order statistics [Bull. Inst. Internat. Statist. 45 (1973) 295-300; Ann. Statist. 2 (1974) 1034-1039]. This intersection grants us different tools for deriving distributional properties. Their comparisons lead to new results, as well as simpler proofs of existing ones. Our main contribution, Theorem 25 in Section 6, describes the asymptotic distribution of the last few terms in a finite i.i.d. size-biased permutation via a Poisson coupling with its few smallest order statistics.Comment: Published at http://dx.doi.org/10.3150/14-BEJ652 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Minimal Embedding Dimensions of Connected Neural Codes

In the past few years, the study of receptive field codes has been of large interest to mathematicians. Here we give a complete characterization of receptive field codes realizable by connected receptive fields and we give the minimal embedding dimensions of these codes. In particular, we show that all connected codes are realizable in dimension at most 3. To our knowledge, this is the first family of receptive field codes for which the exact characterization and minimal embedding dimension is known.Comment: 9 pages, 4 figure

Implicit Causal Models for Genome-wide Association Studies

Progress in probabilistic generative models has accelerated, developing richer models with neural architectures, implicit densities, and with scalable algorithms for their Bayesian inference. However, there has been limited progress in models that capture causal relationships, for example, how individual genetic factors cause major human diseases. In this work, we focus on two challenges in particular: How do we build richer causal models, which can capture highly nonlinear relationships and interactions between multiple causes? How do we adjust for latent confounders, which are variables influencing both cause and effect and which prevent learning of causal relationships? To address these challenges, we synthesize ideas from causality and modern probabilistic modeling. For the first, we describe implicit causal models, a class of causal models that leverages neural architectures with an implicit density. For the second, we describe an implicit causal model that adjusts for confounders by sharing strength across examples. In experiments, we scale Bayesian inference on up to a billion genetic measurements. We achieve state of the art accuracy for identifying causal factors: we significantly outperform existing genetics methods by an absolute difference of 15-45.3%

The Tropical Commuting Variety

We study tropical commuting matrices from two viewpoints: linear algebra and algebraic geometry. In classical linear algebra, there exist various criteria to test whether two square matrices commute. We ask for similar criteria in the realm of tropical linear algebra, giving conditions for two tropical matrices that are polytropes to commute. From the algebro-geometric perspective, we explicitly compute the tropicalization of the classical variety of commuting matrices in dimension 2 and 3.Comment: 14 pages, 4 figure

Extremal edge polytopes

The "edge polytope" of a finite graph G is the convex hull of the columns of its vertex-edge incidence matrix. We study extremal problems for this class of polytopes. For k =2, 3, 5 we determine the maximum number of vertices of k-neighborly edge polytopes up to a sublinear term. We also construct a family of edge polytopes with exponentially-many facets.Comment: Final version; 16 pages, 3 figures. Published in The Electronic Journal of Combinatoric

Blind quantum computation using the central spin Hamiltonian

Blindness is a desirable feature in delegated computation. In the classical setting, blind computations protect the data or even the program run by a server. In the quantum regime, blind computing may also enable testing computational or other quantum properties of the server system. Here we propose a scheme for universal blind quantum computation using a quantum simulator capable of emulating Heisenberg-like Hamiltonians. Our scheme is inspired by the central spin Hamiltonian in which a single spin controls dynamics of a number of bath spins. We show how, by manipulating this spin, a client that only accesses the central spin can effectively perform blind computation on the bath spins. Remarkably, two-way quantum communication mediated by the central spin is sufficient to ensure security in the scheme. Finally, we provide explicit examples of how our universal blind quantum computation enables verification of the power of the server from classical to stabilizer to full BQP computation.Comment: 8 pages, 2 figure

The Block Pseudo-Marginal Sampler

The pseudo-marginal (PM) approach is increasingly used for Bayesian inference in statistical models, where the likelihood is intractable but can be estimated unbiasedly. %Examples include random effect models, state-space models and data subsampling in big-data settings. Deligiannidis et al. (2016) show how the PM approach can be made much more efficient by correlating the underlying Monte Carlo (MC) random numbers used to form the estimate of the likelihood at the current and proposed values of the unknown parameters. Their approach greatly speeds up the standard PM algorithm, as it requires a much smaller number of samples or particles to form the optimal likelihood estimate. Our paper presents an alternative implementation of the correlated PM approach, called the block PM, which divides the underlying random numbers into blocks so that the likelihood estimates for the proposed and current values of the parameters only differ by the random numbers in one block. We show that this implementation of the correlated PM can be much more efficient for some specific problems than the implementation in Deligiannidis et al. (2016); for example when the likelihood is estimated by subsampling or the likelihood is a product of terms each of which is given by an integral which can be estimated unbiasedly by randomised quasi-Monte Carlo. Our article provides methodology and guidelines for efficiently implementing the block PM. A second advantage of the the block PM is that it provides a direct way to control the correlation between the logarithms of the estimates of the likelihood at the current and proposed values of the parameters than the implementation in Deligiannidis et al. (2016). We obtain methods and guidelines for selecting the optimal number of samples based on idealized but realistic assumptions.Comment: 41 pages, 6 tables , 4 figure

Towards stability and optimality in stochastic gradient descent

Iterative procedures for parameter estimation based on stochastic gradient descent allow the estimation to scale to massive data sets. However, in both theory and practice, they suffer from numerical instability. Moreover, they are statistically inefficient as estimators of the true parameter value. To address these two issues, we propose a new iterative procedure termed averaged implicit SGD (AI-SGD). For statistical efficiency, AI-SGD employs averaging of the iterates, which achieves the optimal Cram\'{e}r-Rao bound under strong convexity, i.e., it is an optimal unbiased estimator of the true parameter value. For numerical stability, AI-SGD employs an implicit update at each iteration, which is related to proximal operators in optimization. In practice, AI-SGD achieves competitive performance with other state-of-the-art procedures. Furthermore, it is more stable than averaging procedures that do not employ proximal updates, and is simple to implement as it requires fewer tunable hyperparameters than procedures that do employ proximal updates.Comment: Appears in Artificial Intelligence and Statistics, 201

Stochastic gradient descent methods for estimation with large data sets

We develop methods for parameter estimation in settings with large-scale data sets, where traditional methods are no longer tenable. Our methods rely on stochastic approximations, which are computationally efficient as they maintain one iterate as a parameter estimate, and successively update that iterate based on a single data point. When the update is based on a noisy gradient, the stochastic approximation is known as standard stochastic gradient descent, which has been fundamental in modern applications with large data sets. Additionally, our methods are numerically stable because they employ implicit updates of the iterates. Intuitively, an implicit update is a shrinked version of a standard one, where the shrinkage factor depends on the observed Fisher information at the corresponding data point. This shrinkage prevents numerical divergence of the iterates, which can be caused either by excess noise or outliers. Our sgd package in R offers the most extensive and robust implementation of stochastic gradient descent methods. We demonstrate that sgd dominates alternative software in runtime for several estimation problems with massive data sets. Our applications include the wide class of generalized linear models as well as M-estimation for robust regression

Deep Laplacian Pyramid Network for Text Images Super-Resolution

Convolutional neural networks have recently demonstrated interesting results for single image super-resolution. However, these networks were trained to deal with super-resolution problem on natural images. In this paper, we adapt a deep network, which was proposed for natural images superresolution, to single text image super-resolution. To evaluate the network, we present our database for single text image super-resolution. Moreover, we propose to combine Gradient Difference Loss (GDL) with L1/L2 loss to enhance edges in super-resolution image. Quantitative and qualitative evaluations on our dataset show that adding the GDL improves the super-resolution results.Comment: paper, 6 page