Reliability of Erasure Coded Storage Systems: A Geometric Approach

We consider the probability of data loss, or equivalently, the reliability function for an erasure coded distributed data storage system under worst case conditions. Data loss in an erasure coded system depends on probability distributions for the disk repair duration and the disk failure duration. In previous works, the data loss probability of such systems has been studied under the assumption of exponentially distributed disk failure and disk repair durations, using well-known analytic methods from the theory of Markov processes. These methods lead to an estimate of the integral of the reliability function. Here, we address the problem of directly calculating the data loss probability for general repair and failure duration distributions. A closed limiting form is developed for the probability of data loss and it is shown that the probability of the event that a repair duration exceeds a failure duration is sufficient for characterizing the data loss probability. For the case of constant repair duration, we develop an expression for the conditional data loss probability given the number of failures experienced by a each node in a given time window. We do so by developing a geometric approach that relies on the computation of volumes of a family of polytopes that are related to the code. An exact calculation is provided and an upper bound on the data loss probability is obtained by posing the problem as a set avoidance problem. Theoretical calculations are compared to simulation results.Comment: 28 pages. 8 figures. Presented in part at IEEE International Conference on BigData 2013, Santa Clara, CA, Oct. 2013 and to be presented in part at 2014 IEEE Information Theory Workshop, Tasmania, Australia, Nov. 2014. New analysis added May 2015. Further Update Aug. 201

On the Interactive Communication Cost of the Distributed Nearest Lattice Point Problem

We consider the problem of distributed computation of the nearest lattice point for a two dimensional lattice. An interactive two-party model of communication is considered. Algorithms with bounded, as well as unbounded, number of rounds of communication are considered. For the algorithm with a bounded number of rounds, expressions are derived for the error probability as a function of the total number of communicated bits. We observe that the error exponent depends on the lattice. With an infinite number of allowed communication rounds, the average cost of achieving zero error probability is shown to be finite.Comment: Extended (journal version) of a previous submission. arXiv admin note: text overlap with arXiv:1701.0845

Multiple Description Vector Quantization with Lattice Codebooks: Design and Analysis

The problem of designing a multiple description vector quantizer with lattice codebook Lambda is considered. A general solution is given to a labeling problem which plays a crucial role in the design of such quantizers. Numerical performance results are obtained for quantizers based on the lattices A_2 and Z^i, i=1,2,4,8, that make use of this labeling algorithm. The high-rate squared-error distortions for this family of L-dimensional vector quantizers are then analyzed for a memoryless source with probability density function p and differential entropy h(p) < infty. For any a in (0,1) and rate pair (R,R), it is shown that the two-channel distortion d_0 and the channel 1 (or channel 2) distortions d_s satisfy lim_{R -> infty} d_0 2^(2R(1+a)) = (1/4) G(Lambda) 2^{2h(p)} and lim_{R -> infty} d_s 2^(2R(1-a)) = G(S_L) 2^2h(p), where G(Lambda) is the normalized second moment of a Voronoi cell of the lattice Lambda and G(S_L) is the normalized second moment of a sphere in L dimensions.Comment: 46 pages, 14 figure

A Zador-Like Formula for Quantizers Based on Periodic Tilings

We consider Zador's asymptotic formula for the distortion-rate function for a variable-rate vector quantizer in the high-rate case. This formula involves the differential entropy of the source, the rate of the quantizer in bits per sample, and a coefficient G which depends on the geometry of the quantizer but is independent of the source. We give an explicit formula for G in the case when the quantizing regions form a periodic tiling of n-dimensional space, in terms of the volumes and second moments of the Voronoi cells. As an application we show, extending earlier work of Kashyap and Neuhoff, that even a variable-rate three-dimensional quantizer based on the ``A15'' structure is still inferior to a quantizer based on the body-centered cubic lattice. We also determine the smallest covering radius of such a structure.Comment: 8 page

Constructive spherical codes on layers of flat tori

A new class of spherical codes is constructed by selecting a finite subset of flat tori from a foliation of the unit sphere S^{2L-1} of R^{2L} and designing a structured codebook on each torus layer. The resulting spherical code can be the image of a lattice restricted to a specific hyperbox in R^L in each layer. Group structure and homogeneity, useful for efficient storage and decoding, are inherited from the underlying lattice codebook. A systematic method for constructing such codes are presented and, as an example, the Leech lattice is used to construct a spherical code in R^{48}. Upper and lower bounds on the performance, the asymptotic packing density and a method for decoding are derived.Comment: 9 pages, 5 figures, submitted to IEEE Transactions on Information Theor

Molecular Technique to Understand Deep Microbial Diversity

Current sequencing-based and DNA microarray techniques to study microbial diversity are based on an initial PCR (polymerase chain reaction) amplification step. However, a number of factors are known to bias PCR amplification and jeopardize the true representation of bacterial diversity. PCR amplification of the minor template appears to be suppressed by the exponential amplification of the more abundant template. It is widely acknowledged among environmental molecular microbiologists that genetic biosignatures identified from an environment only represent the most dominant populations. The technological bottleneck has overlooked the presence of the less abundant minority population, and underestimated their role in the ecosystem maintenance. To generate PCR amplicons for subsequent diversity analysis, bacterial l6S rRNA genes are amplified by PCR using universal primers. Two distinct PCR regimes are employed in parallel: one using normal and the other using biotinlabeled universal primers. PCR products obtained with biotin-labeled primers are mixed with streptavidin-labeled magnetic beads and selectively captured in the presence of a magnetic field. Less-abundant DNA templates that fail to amplify in this first round of PCR amplification are subjected to a second round of PCR using normal universal primers. These PCR products are then subjected to downstream diversity analyses such as conventional cloning and sequencing. A second round of PCR amplified the minority population and completed the deep diversity picture of the environmental sample