Performance Analysis of Grammar-Based Codes Revisited

Communications in Information and Systems, 2002

A grammar-based code losslessly compresses each finite-alphabet data string x by compressing a context-free grammar G x which represents x in the sense that the language of G x is {x}. In an earlier paper, we showed that if the grammar G x is a type of grammar called irreducible grammar for every data string x, then the resulting grammar-based code has maximal redundancy/sample O(log log n/ log n) for n data samples. To further reduce the maximal redundancy/sample, in the present paper, we first decompose a context-free grammar into its structure and its data content, then encode the data content conditional on the structure, and finally replace the irreducible grammar condition with a mild condition on the structures of all grammars used to represent distinct data strings of a fixed length. The resulting grammar-based codes are called structured grammar-based codes. We prove a coding theorem which shows that a structured grammar-based code has maximal redundancy/sample O(1/ log n) provided that a weak regular structure condition is satisfied.

IEEE Transactions on Information Theory, 2000

We i n vestigate a type of lossless source code called a grammar based code, which, in response to any input data string x over a xed nite alphabet, selects a context-free grammar G x representing x in the sense that x is the unique string belonging to the language generated by G x . Lossless compression of x takes place indirectly via compression of the production rules of the grammar G x . I t i s s h o wn that, subject to some mild restrictions, a grammar based code is a universal code with respect to the family of nite state information sources over the nite alphabet. Redundancy bounds for grammar based codes are established. Reduction rules for designing grammar based codes are presented.

IEEE Transactions on Information Theory, 2003

A grammar transform is a transformation that converts any data sequence to be compressed into a grammar from which the original data sequence can be fully reconstructed. In a grammar-based code, a data sequence is first converted into a grammar by a grammar transform and then losslessly encoded. In this paper, a greedy grammar transform is first presented; this grammar transform constructs sequentially a sequence of irreducible grammars from which the original data sequence can be recovered incrementally. Based on this grammar transform, three universal lossless data compression algorithms, a sequential algorithm, an improved sequential algorithm, and a hierarchical algorithm, are then developed. These algorithms combine the power of arithmetic coding with that of string matching. It is shown that these algorithms are all universal in the sense that they can achieve asymptotically the entropy rate of any stationary, ergodic source. Moreover, it is proved that their worst case redundancies among all individual sequences of length are upper-bounded by log log log , where is a constant.

Manufacturing Engineer, 2004

Grammar-based c o d i n g i s i n v e s t i g a t e d f r o m three new perspectives. F i r s t , w e revisit the perf o r m a n c e a n a l y s i s o f grammar-based c o d e s by proposi n g context-based run-length e n c o d i n g a l g o r i t h m s as n e w p e r f o r m a n c e b e n c h m a r k s. A r e d u n d a n c y r e s u l t stronger than a l l previous c o r r e s p o n d i n g results is established. We then extend the a n a l y s i s of g r a m m a rbased c o d e s to sources w i t h c o u n t a b l y infinite a l p h abets. L e t A denote a n a r b i t r a r y class of s t a t i o n a r y , e r g o d i c sources w i t h a c o u n t a b l y infinite a l p h a b e t. It is s h o w n that grammar-based codes can he modified so that they are universal with respect to any A for w h i c h there e x i s t s a universal code. Moreover, u p p e r b o u n d s o n t h e worst-case r e d u n d a n c i e s of grammarbased codes among Large sets o f length-n i n d i v i d u a l s e q u e n c e s f r o m a c o u n t a b l y infinite a l p h a b e t are e s tablished. Finally, we propose a n e w t h e o r e t i c framew o r k f o r c o m p r e s s i o n i n w h i c h grammars rather than s e q u e n t i a l s t o c h a s t i c processes are used as source gene r a t i n g models, and p o i n t out some open p r o b l e m s i n the framework. 'Unless otherwise specified, throughout this paper we focus our discussion on grmmilr-based codes that transform each data Sequence into an irreducible context-free grammar, and will simply refer to them ill grammar-based codes. bet 111, PI. 210g stands for the logarithm to base 2 throughout this paper.

Proceedings of the 1999 IEEE Information Theory and Communications Workshop (Cat. No. 99EX253), 1999

A n e w universal lossless source coding theory is presented. Within this theory, a lossless source code called a grammar based code first transforms the original data sequence to be compressed into a context free grammar, from which the original data sequence can be fully reconstructed b y performing parallel substitutions, and then uses an arithmetic coding algorithm to compress the context free gramm a r or the corresponding sequence of parsed phrases. It is shown that if a grammar-based code transforms each data sequence into an irreducible context free grammar, then the grammar-based code is universal for the class of stationary, ergodic sources. Specific redundancy bounds are also given. 'This work was supported in part by the Natural Sciences and Engineering Research Council of Canada under Grant RGPIN203035-98, the Communications and Information Technology Ontario, and the National Sciences Foundation under Grant NCR-9627965.

IEEE Transactions on Information Theory, 2001

ACM Transactions on Information Systems, 1990

Text compression is of considerable theoretical and practical interest. It is, for example, becoming increasingly important for satisfying the requirements of tting a large database onto a single CD-ROM. Many of the compression techniques discussed in the literature are model based. We here propose the notion of a formal grammar as a exible model of text generation that encompasses most of the models o ered before as well as, in principle, extending the possibility of compression to a much more general class of languages. Assuming a general model of text generation, a derivation is given of the well known Shannon entropy f o r m ula, making possible a theory of information based upon text representation rather than on communication. The ideas are shown to apply to a number of commonly used text models. Finally, w e focus on a Markov model of text generation, suggest an information theoretic measure of similarity b e t ween two probability distributions, and develop a clustering algorithm based on this measure. This algorithm allows us to cluster Markov states, and thereby base our compression algorithm on a smaller number of probability distributions than would otherwise have been required. A number of theoretical consequences of this approach to compression are explored, and a detailed example is given. * This does not foreclose the possibility of generating a number of messages in sequence. However, when we do so, we think of the messages as being generated independently, and the encoding and decoding processes starting over again each time. (This contrasts with the notion of a code extension, i n w h i c h, for encoding purposes, a xed number of successive independently generated messages is treated as a single message from a correspondingly large message set 8].

IEEE Transactions on Information Theory, 1996

A general class of sequential codes for lossless compression of individual sequences on a finite alphabet is defined, including many types of codes that one would want to implement. The principal requirement for membership in the class is that the encoding and decoding operations be performable on a computer. The OPTA function for the class of codes is then considered, which is the function that assigns to each individual sequence the infimum of the rates at which the sequence can be compressed over this class of sequential codes. Two results about the OPTA function are obtained: 1) it is shown that any sequential code in the class compresses some individual sequence at a rate strictly greater than the rate for that sequence given by the OPTA function; and 2) it is shown that the OPTA function takes a value strictly greater than that of the Kolmogorov complexity rate function for some individual sequences.

IEEE Transactions on Information Theory, 2000

We consider the problem of lossless compression of binary trees, with the aim of reducing the number of code bits needed to store or transmit such trees. A lossless grammarbased code is presented, which encodes each binary tree into a binary codeword in two steps. In the first step, the tree is transformed into a context-free grammar from which the tree can be reconstructed. In the second step, the context-free grammar is encoded into a binary codeword. The decoder of the grammar-based code decodes the original tree from its codeword by reversing the two encoding steps. It is shown that the resulting grammar-based binary tree compression code is a universal code on a family of probabilistic binary tree source models satisfying certain weak restrictions.

Theoretical Computer Science, 2012

In this paper, the optimality proof of Ziv-Lempel coding is re-studied, and a more general compression optimality theorem is derived. In particular, the property of quasi-distinct parsing is defined. This property allows infinitely many repetitions of phrases in the parsing as long as the total number of repetitions is o(n/ log n), where n is length of the parsed string. The quasi-distinct parsing property is weaker than distinct parsing used in the original proof which does not allow repetitions of phrases in the parsing. Yet we show that the theorem holds with this weaker property as well. This provides a better understanding of the optimality proof of Ziv-Lempel coding, together with a new tool for proving optimality of other compression schemes which is applicable for a much wider family of codes. To demonstrate the possible use of this generalization, a new coding method-the Arithmetic Progression Tree coding (APT)-is presented. This new coding method is based on a principle that is very different from Ziv-Lempel's coding. Nevertheless, the APT coding is analyzed in this paper and using the generalized theorem shown to be asymptotically optimal up to a constant factor, 1 if the APT quasi-distinctness hypothesis holds. An empirical evidence that this hypothesis holds is also given.