Variable-Length Codes for Sources With Equiprobable Symbols

2007

Abstract We consider the problem of choosing a block-length to achieve a desired probability of error for fixed-length lossless source coding and channel coding of a finite amount of payload data. This is closely related to the issue of redundancy. While Baron, et al.

2007

We offer novel algorithms for efficient encoding/decoding of variable-to-fixed length codes, requiring at most quadratic amount of space: O(L 2), where L is the depth of a coding tree. This is a major improvement compared to exponential O(2 L) usage of space by conventional techniques using complete representations of coding trees in computer's memory. These savings are achieved by utilizing algebraic properties of VF coding trees constructed by using Tunstall or Khodak algorithms, and employing combinatorial enumeration techniques for encoding/decoding of codewords. The encoding/decoding complexity of our algorithms is linear with the number of symbols they process. As a side product, we also derive an exact formulae for the average redundancy of such codes under memoryless sources, and show its usefulness for analysis and design of codes with small number of codewords.

Problems of Information Transmission, 2012

The compression-complexity trade-off of lossy compression algorithms that are based on a random codebook or a random database is examined. Motivated, in part, by recent results of Gupta-Verdú-Weissman (GVW) and their underlying connections with the pattern-matching scheme of Kontoyiannis' lossy Lempel-Ziv algorithm, we introduce a non-universal version of the lossy Lempel-Ziv method (termed LLZ). The optimality of LLZ for memoryless sources is established, and its performance is compared to that of the GVW divide-and-conquer approach. Experimental results indicate that the GVW approach often yields better compression than LLZ, but at the price of much higher memory requirements. To combine the advantages of both, we introduce a hybrid algorithm (HYB) that utilizes both the divide-and-conquer idea of GVW and the single-database structure of LLZ. It is proved that HYB shares with GVW the exact same rate-distortion performance and implementation complexity, while, like LLZ, requiring less memory, by a factor which may become unbounded, depending on the choice of the relevant design parameters. Experimental results are also presented, illustrating the performance of all three methods on data generated by simple discrete memoryless sources. In particular, the HYB algorithm is shown to outperform existing schemes for the compression of some simple discrete sources with respect to the Hamming distortion criterion. 2 codes, turbo codes, and local message-passing decoding algorithms; see, e.g., [33][48][34], the texts [16][32][44], and the references therein.

1995

Two construction techniques for variable-length error-correcting (VLEC) codes are given. The first uses fixed-length linear codes and anticodes to build new VLEC codes, whereas the second uses a heuristic algorithm to perform a computer search for good VLEC codes. VLEC codes may be used for combined source and channel coding. It is shown that over an additive white Gaussian noise channel the codes so constructed can perform better than standard cascaded source and channel codes with similar parameters.

CODING ALEX NANA REMEGIO, JR.

Communications of the ACM, 1987

2009

We establish a general framework for construction of small ensembles of capacity achieving linear codes for a wide range of (not necessarily memoryless) discrete symmetric channels, and in particular, the binary erasure and symmetric channels. The main tool used in our constructions is the notion of randomness extractors and lossless condensers that are regarded as central tools in theoretical computer science. Same as random codes, the resulting ensembles preserve their capacity achieving properties under any change of basis. Using known explicit constructions of condensers, we obtain specific ensembles whose size is as small as polynomial in the block length. By applying our construction to Justesen's concatenation scheme (Justesen, 1972) we obtain explicit capacity achieving codes for BEC (resp., BSC) with almost linear time encoding and almost linear time (resp., quadratic time) decoding and exponentially small error probability.

arXiv: Information Theory, 2015

We propose almost instantaneous fixed-to-variable-length (AIFV) codes such that two (resp. K − 1) code trees are used if code symbols are binary (resp. K-ary for K ≥ 3), and source symbols are assigned to incomplete internal nodes in addition to leaves. Although the AIFV codes are not instantaneous codes, they are devised such that the decoding delay is at most two bits (resp. one code symbol) in the case of binary (resp. K-ary) code alphabet. The AIFV code can attain better average compression rate than the Huffman code at the expenses of a little decoding delay and a little large memory size to store multiple code trees. We also show for the binary and ternary AIFV codes that the optimal AIFV code can be obtained by solving 0-1 integer programming problems. Index Terms AIFV code, Huffman code, FV code, code tree, Kraft inequality, Integer programming I. INTRODUCTION Lossless source codes are classified into fixed-to-variable-length (FV) codes and variable-to-fixed-length (VF) codes, which can be represented by code trees and parse trees, respectively. It is well known that the Huffman coding [1] and Tunstall coding [2] can attain the best compression rate in FV codes and VF codes, respectively, for stationary memoryless sources if a single code tree or a single parse tree is used. But, Yamamoto and Yokoo [3] showed that the AIVF (almost instantaneous variable-to-fixed length) coding can attain better compression rate than the Tunstall coding. An AIVF code uses |X |−1 parse trees for a source alphabet X and codewords are assigned to incomplete internal nodes in addition to leaves in each parse tree. Although instantaneous encoding is not possible since incomplete internal nodes are used for encoding, the AIVF code is devised such that the encoding delay is at most one source symbol, and hence the code is called almost instantaneous. Furthermore, Yoshida and Kida [4][5] showed that any AIVF code can be encoded and decoded by a single virtual multiple parse tree and the total number of nodes can be considerably reduced by the integration. In the case of FV codes, it is well known by Kraft and McMillan Theorems [6][7][8] that any uniquely decodable FV code must satisfy Kraft's inequality, and such a code can be realized by an instantaneous FV