New Non-asymptotic Random Channel Coding Theorems

New non-asymptotic random coding theorems (with error probability $\epsilon$ and finite block length $n$) based on Gallager parity check ensemble and Shannon random code ensemble with a fixed codeword type are established for discrete input arbitrary output channels. The resulting non-asymptotic achievability bounds, when combined with non-asymptotic equipartition properties developed in the paper, can be easily computed. Analytically, these non-asymptotic achievability bounds are shown to be asymptotically tight up to the second order of the coding rate as $n$ goes to infinity with either constant or sub-exponentially decreasing $\epsilon$. Numerically, they are also compared favourably, for finite $n$ and $\epsilon$ of practical interest, with existing non-asymptotic achievability bounds in the literature in general.Comment: 48 pages and 12 figure

Jar Decoding: Non-Asymptotic Converse Coding Theorems, Taylor-Type Expansion, and Optimality

Recently, a new decoding rule called jar decoding was proposed; under jar decoding, a non-asymptotic achievable tradeoff between the coding rate and word error probability was also established for any discrete input memoryless channel with discrete or continuous output (DIMC). Along the path of non-asymptotic analysis, in this paper, it is further shown that jar decoding is actually optimal up to the second order coding performance by establishing new non-asymptotic converse coding theorems, and determining the Taylor expansion of the (best) coding rate $R_n (\epsilon)$ of finite block length for any block length $n$ and word error probability $\epsilon$ up to the second order. Finally, based on the Taylor-type expansion and the new converses, two approximation formulas for $R_n (\epsilon)$ (dubbed "SO" and "NEP") are provided; they are further evaluated and compared against some of the best bounds known so far, as well as the normal approximation of $R_n (\epsilon)$ revisited recently in the literature. It turns out that while the normal approximation is all over the map, i.e. sometime below achievable bounds and sometime above converse bounds, the SO approximation is much more reliable as it is always below converses; in the meantime, the NEP approximation is the best among the three and always provides an accurate estimation for $R_n (\epsilon)$. An important implication arising from the Taylor-type expansion of $R_n (\epsilon)$ is that in the practical non-asymptotic regime, the optimal marginal codeword symbol distribution is not necessarily a capacity achieving distribution.Comment: submitted to IEEE Transaction on Information Theory in April, 201

On Linear Operator Channels over Finite Fields

Motivated by linear network coding, communication channels perform linear operation over finite fields, namely linear operator channels (LOCs), are studied in this paper. For such a channel, its output vector is a linear transform of its input vector, and the transformation matrix is randomly and independently generated. The transformation matrix is assumed to remain constant for every T input vectors and to be unknown to both the transmitter and the receiver. There are NO constraints on the distribution of the transformation matrix and the field size. Specifically, the optimality of subspace coding over LOCs is investigated. A lower bound on the maximum achievable rate of subspace coding is obtained and it is shown to be tight for some cases. The maximum achievable rate of constant-dimensional subspace coding is characterized and the loss of rate incurred by using constant-dimensional subspace coding is insignificant. The maximum achievable rate of channel training is close to the lower bound on the maximum achievable rate of subspace coding. Two coding approaches based on channel training are proposed and their performances are evaluated. Our first approach makes use of rank-metric codes and its optimality depends on the existence of maximum rank distance codes. Our second approach applies linear coding and it can achieve the maximum achievable rate of channel training. Our code designs require only the knowledge of the expectation of the rank of the transformation matrix. The second scheme can also be realized ratelessly without a priori knowledge of the channel statistics.Comment: 53 pages, 3 figures, submitted to IEEE Transaction on Information Theor

Capacity Analysis of Linear Operator Channels over Finite Fields

Motivated by communication through a network employing linear network coding, capacities of linear operator channels (LOCs) with arbitrarily distributed transfer matrices over finite fields are studied. Both the Shannon capacity $C$ and the subspace coding capacity $C_{\text{SS}}$ are analyzed. By establishing and comparing lower bounds on $C$ and upper bounds on $C_{\text{SS}}$, various necessary conditions and sufficient conditions such that $C=C_{\text{SS}}$ are obtained. A new class of LOCs such that $C=C_{\text{SS}}$ is identified, which includes LOCs with uniform-given-rank transfer matrices as special cases. It is also demonstrated that $C_{\text{SS}}$ is strictly less than $C$ for a broad class of LOCs. In general, an optimal subspace coding scheme is difficult to find because it requires to solve the maximization of a non-concave function. However, for a LOC with a unique subspace degradation, $C_{\text{SS}}$ can be obtained by solving a convex optimization problem over rank distribution. Classes of LOCs with a unique subspace degradation are characterized. Since LOCs with uniform-given-rank transfer matrices have unique subspace degradations, some existing results on LOCs with uniform-given-rank transfer matrices are explained from a more general way.Comment: To appear in IEEE Transactions on Information Theor