Generalized Belief Propagation

2012 International Conference on Control, Automation and Information Sciences (ICCAIS), 2012

Numerous inference problems in statistical physics, computer vision or error-correcting coding theory consist in approximating the marginal probability distributions on Markov Random Fields (MRF). The Belief Propagation (BP) is an accurate solution that is optimal if the MRF is loop free and suboptimal otherwise. In the context of error-correcting coding theory, any Low-Density Parity-Check (LDPC) code has a graphical representation, the Tanner graph, which is a particular MRF. It is used as a media for the BP algortithm to correct the bits, damaged by a noisy channel, by estimating their probability distributions. Though loops and combination thereof in the Tanner graph prevent the BP from being optimal, especially harmful topological structures called the trappingsets. The BP has been extended to the Generalized Belief Propagation (GBP). This message-passing algorithm runs on a non unique mapping of the Tanner graph, namely the regiongraph, such that its nodes are gatherings of the Tanner graph nodes. Then it appears the possibility to decrease the loops effect, making the GBP more accurate than the BP. In this article, we expose a novel region graph construction suited to the Tanner code, an LDPC code whose Tanner graph is entirely covered by trapping-sets. Furthermore, we investigate the dynamic behavior of the GBP compared with that of the BP to understand its evolution in terms of the Signal-to-Noise Ratio (SNR). To this end we make use of classical estimators and we introduce a new one called the hyperspheres method.

IEEE Communications Letters, 2004

In this letter, we propose two modifications to belief propagation (BP) decoding algorithm. The modifications are based on reducing the reliability of messages throughout the iteration process, and are particularly effective for short low-density parity-check codes, where the existence of cycles makes the original BP algorithm perform suboptimal. The proposed algorithms, referred to as "normalized BP" and "offset BP," reduce the absolute value of the outgoing log-likelihood ratio messages at variable nodes by using a multiplicative factor and an additive factor, respectively. Simulation results show that both algorithms perform more or less the same, and both outperform BP in error performance.

2009 IEEE International Symposium on Information Theory, 2009

The convergence of iterative decoding schemes utilizing belief propagation is considered. A quantitative bound for the output L-values of a Turbo decoder is given that only depends on the received word and thus is independent from the decoder iterations. Further, it is shown that the exponential increase of the L-values in each iteration within an LDPC decoder is limited by the degree of the variable nodes.

2012

In the context of channel coding, the Generalized Belief Propagation (GBP) is an iterative algorithm used to recover the transmission bits sent through a noisy channel. To ensure a reliable transmission, we apply a map on the bits, that is called a code. This code induces artificial correlations between the bits to send, and it can be modeled by a graph whose nodes are the bits and the edges are the correlations. This graph, called Tanner graph, is used for most of the decoding algorithms like Belief Propagation or Gallager-B. The GBP is based on a non unic transformation of the Tanner graph into a so called region-graph. A clear advantage of the GBP over the other algorithms is the freedom in the construction of this graph. In this article, we explain a particular construction for specific graph topologies that involves relevant performance of the GBP. Moreover, we investigate the behavior of the GBP considered as a dynamic system in order to understand the way it evolves in terms ...

2010 IEEE International Conference on Acoustics, Speech and Signal Processing, 2010

Belief propagation is known to perform extremely well in many practical statistical inference and learning problems using graphical models, even in the presence of multiple loops. The iterative use of belief propagation algorithm on loopy graphs is referred to as Loopy Belief Propagation (LBP). Various sufficient conditions for convergence of LBP have been presented; however, general necessary conditions for its convergence to a unique fixed point remain unknown. Because the approximation of beliefs to true marginal probabilities has been shown to relate to the convergence of LBP, several methods have been explored whose aim is to obtain distance bounds on beliefs when LBP fails to converge. In this paper, we derive uniform and non-uniform error bounds on messages, which are tighter than existing ones in literature, and use these bounds to derive sufficient conditions for the convergence of LBP in terms of the sum-product algorithm. We subsequently use these bounds to study the dynamic behavior of the sum-product algorithm, and analyze the relation between convergence of LBP and sparsity and walk-summability of graphical models. We finally use the bounds derived to investigate the accuracy of LBP, as well as the scheduling priority in asynchronous LBP.

IEEE Information Theory Workshop 2010 (ITW 2010), 2010

This paper proposes a new algorithm for the linear least squares problem where the unknown variables are constrained to be in a finite set. The factor graph that corresponds to this problem is very loopy; in fact, it is a complete graph. Hence, applying the Belief Propagation (BP) algorithm yields very poor results. The Pseudo Prior Belief Propagation (PPBP) algorithm is a variant of the BP algorithm that can achieve near maximum likelihood (ML) performance with low computational complexity. First, we use the minimum mean square error (MMSE) detection to yield a pseudo prior information on each variable. Next we integrate this information into a loopy Belief Propagation (BP) algorithm as a pseudo prior. We show that, unlike current paradigms, the Belief Propagation (BP) algorithm can be advantageous even for dense graphs with many short loops. The performance of the proposed algorithm is demonstrated on the MIMO detection problem based on simulation results.

2014 IEEE Information Theory Workshop (ITW 2014), 2014

A low-density parity-check (LDPC) code is a linear block code described by a sparse parity-check matrix, which can be efficiently represented by a bipartite Tanner graph. The standard iterative decoding algorithm, known as belief propagation, passes messages along the edges of this Tanner graph. Density evolution is an efficient method to analyze the performance of the belief propagation decoding algorithm for a particular LDPC code ensemble, enabling the determination of a decoding threshold. The basic problem addressed in this work is how to optimize the Tanner graph so that the decoding threshold is as large as possible. We introduce a new code optimization technique which involves the search space range which can be thought of as minimizing randomness in differential evolution or limiting the search range in exhaustive search. This technique is applied to the design of good irregular LDPC codes and multiedge type LDPC codes.

2016

Belief propagation is known to perform extremely well in many practical statistical inference and learning problems using graphical models, even in the presence of multiple loops. The iterative use of belief propagation algorithm on loopy graphs is referred to as Loopy Belief Propagation (LBP). Various sufficient conditions for convergence of LBP have been presented; however, general necessary conditions for its convergence to a unique fixed point remain unknown. Because the approximation of beliefs to true marginal probabilities has been shown to relate to the convergence of LBP, several methods have been explored whose aim is to obtain distance bounds on beliefs when LBP fails to converge. In this paper, we derive uniform and non-uniform error bounds on messages, which are tighter than existing ones in literature, and use these bounds to derive sufficient conditions for the convergence of LBP in terms of the sum-product algorithm. We subsequently use these bounds to study the dynamic behavior of the sum-product algorithm, and analyze the relation between convergence of LBP and sparsity and walk-summability of graphical models. We finally use the bounds derived to investigate the accuracy of LBP, as well as the scheduling priority in asynchronous LBP.

2010

Belief propagation is known to perform extremely well in many practical statistical inference and learning problems using graphical models, even in the presence of multiple loops. The iterative use of belief propagation algorithm on loopy graphs is referred to as Loopy Belief Propagation (LBP). Various sufficient conditions for convergence of LBP have been presented; however, general necessary conditions for its convergence to a unique fixed point remain unknown. Because the approximation of beliefs to true marginal probabilities has been shown to relate to the convergence of LBP, several methods have been explored whose aim is to obtain distance bounds on beliefs when LBP fails to converge. In this paper, we derive uniform and non-uniform error bounds on messages, which are tighter than existing ones in literature, and use these bounds to derive sufficient conditions for the convergence of LBP in terms of the sum-product algorithm. We subsequently use these bounds to study the dynamic behavior of the sum-product algorithm, and analyze the relation between convergence of LBP and sparsity and walk-summability of graphical models. We finally use the bounds derived to investigate the accuracy of LBP, as well as the scheduling priority in asynchronous LBP.

An important part of problems in statistical physics and computer science can be expressed as the computation of marginal probabilities over a Markov Random Field. The belief propagation algorithm, which is an exact procedure to compute these marginals when the underlying graph is a tree, has gained its popularity as an efficient way to approximate them in the more general case. In this paper, we focus on an aspect of the algorithm that did not get that much attention in the literature, which is the effect of the normalization of the messages. We show in particular that, for a large class of normalization strategies, it is possible to focus only on belief convergence. Following this, we express the necessary and sufficient conditions for local stability of a fixed point in terms of the graph structure and the beliefs values at the fixed point. We also explicit some connexion between the normalization constants and the underlying Bethe Free Energy.