Feedback and belief propagation

2006

Abstract We demonstrate that feedback in discrete memoryless channels has the capability of greatly lowering the block error rate of codes designed for open-loop operation. First we show how to use full feedback of the channel output to turn any capacity achieving code into a reliability-function achieving code. Second, we propose a practical embodiment based on sparse-graph codes, belief propagation, and a variation of the closed-loop iterative doping algorithm.

SIAM Journal on Discrete Mathematics, 2011

The subject of this paper is transmission over a general class of binary-input memoryless symmetric channels using error correcting codes based on sparse graphs, namely low-density generator-matrix and low-density parity-check codes. The optimal (or ideal) decoder based on the posterior measure over the code bits, and its relationship to the sub-optimal belief propagation decoder, are investigated. We consider the correlation (or covariance) between two codebits, averaged over the noise realizations, as a function of the graph distance, for the optimal decoder. Our main result is that this correlation decays exponentially fast for fixed general low-density generator-matrix codes and high enough noise parameter, and also for fixed general lowdensity parity-check codes and low enough noise parameter. This has many consequences. Appropriate performance curves -called GEXIT functions -of the belief propagation and optimal decoders match in high/low noise regimes. This means that in high/low noise regimes the performance curves of the optimal decoder can be computed by density evolution. Another interpretation is that the replica predictions of spin-glass theory are exact. Our methods are rather general and use cluster expansions first developed in the context of mathematical statistical mechanics.

2014 IEEE Information Theory Workshop (ITW 2014), 2014

A low-density parity-check (LDPC) code is a linear block code described by a sparse parity-check matrix, which can be efficiently represented by a bipartite Tanner graph. The standard iterative decoding algorithm, known as belief propagation, passes messages along the edges of this Tanner graph. Density evolution is an efficient method to analyze the performance of the belief propagation decoding algorithm for a particular LDPC code ensemble, enabling the determination of a decoding threshold. The basic problem addressed in this work is how to optimize the Tanner graph so that the decoding threshold is as large as possible. We introduce a new code optimization technique which involves the search space range which can be thought of as minimizing randomness in differential evolution or limiting the search range in exhaustive search. This technique is applied to the design of good irregular LDPC codes and multiedge type LDPC codes.

IEEE Transactions on Information Theory, 2000

Variable-length block-coding schemes are investigated for discrete memoryless channels with ideal feedback under cost constraints. Upper and lower bounds are found for the minimum achievable probability of decoding error Pe;min as a function of constraints R; P; and on the transmission rate, average cost, and average block length, respectively. For given R and P, the lower and upper bounds to the exponent 0(ln Pe;min)= are asymptotically equal as ! 1. The resulting reliability function, lim !1 (0 ln Pe;min)= , as a function of R and P, is concave in the pair (R; P) and generalizes the linear reliability function of Burnashev to include cost constraints. The results are generalized to a class of discrete-time memoryless channels with arbitrary alphabets, including additive Gaussian noise channels with amplitude and power constraints.

2012

In the context of channel coding, the Generalized Belief Propagation (GBP) is an iterative algorithm used to recover the transmission bits sent through a noisy channel. To ensure a reliable transmission, we apply a map on the bits, that is called a code. This code induces artificial correlations between the bits to send, and it can be modeled by a graph whose nodes are the bits and the edges are the correlations. This graph, called Tanner graph, is used for most of the decoding algorithms like Belief Propagation or Gallager-B. The GBP is based on a non unic transformation of the Tanner graph into a so called region-graph. A clear advantage of the GBP over the other algorithms is the freedom in the construction of this graph. In this article, we explain a particular construction for specific graph topologies that involves relevant performance of the GBP. Moreover, we investigate the behavior of the GBP considered as a dynamic system in order to understand the way it evolves in terms ...

2008

Physical Communication, 2019

The design of block codes for short information blocks (e.g., a thousand or less information bits) is an open research problem that is gaining relevance thanks to emerging applications in wireless communication networks. In this paper, we review some of the most promising code constructions targeting the short block regime, and we compare them with both finite-length performance bounds and classical error-correction coding schemes. The work addresses the use of both binary and high-order modulations over the additive white Gaussian noise channel. We will illustrate how to effectively approach the theoretical bounds with various performance versus decoding complexity tradeoffs.

Journal of Electrical and Computer Engineering, 2014

A novel doping bits based belief propagation decoding algorithm, for rate-adaptive LDPC codes based on fixed bipartite graph code, is proposed. The proposed work modifies the decoding algorithm, by converting the puncturing nodes to regular source nodes and by following the encoding rule at the decoder. The transmitted doping bits in place of punctured bits, with the modified decoding algorithm at the decoder, feed all the punctured nodes with reliable log likelihood ratios. This enables the proposed decoding algorithm to recover all punctured nodes in the early iteration. The fast convergence leads to decoder complexity reduction while providing considerable improvement in performance.

1997

We construct new families of error-correcting codes based on Gallager's low- density parity-check codes, which we call irregular codes. When decoded using belief propagation, our codes can correct more errors than previously known low-density parity-check codes. For example, for rate 1/4 codes on 16,000 bits over a binary symmetric channel, previous low-density parity-check codes can correct up to approximately 16%

2014

variable-rate coding schemes is developed for communication over discrete memoryless channels with noiseless feedback. Algorithms for encoding and decoding that require computations growing linearly with the number of channel inputs used are developed. The error exponent associated with the scheme is shown to be optimal and implies that capacity is achievable. Simulations are performed and support the analytically predicted high performance and low complexity. Index Terms—Error-correction coding, feedback channels, iterative coding. I.