Lossless data compression algorithms based on substitution tables
John Kieffer1998, Conference Proceedings. IEEE Canadian Conference on Electrical and Computer Engineering (Cat. No.98TH8341)
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
This paper presents a class of new lossless data compression algorithms. Each algorithm in this class first transforms the original data to be compressed into an irreducible table representation and then uses an arithmetic code to compress the irreducible table representation. F'rom the irreducible table representation, one can fully reconstruct the original data by performing multistage parallel substitution. A set of rules is described on how to perform hierarchical transformations from the original data to irreducible table representations. Theoretically, it is proved that all these algorithms outperform any finite state sequential compression algorithm and hence achieve the ultimate compression rate for any stationary and ergodic soupce. Furthermore, experiments on several standard images show that even a simple algorithm in this class, the so-called multi-level pattern matching algorithm, outperforms the Lempel-Ziv algorithms and arithmetic codes.
Related papers
Lossless Image Compression Based on Multiple-Tables Arithmetic Coding
Mathematical Problems in Engineering, 2009
This paper is intended to present a lossless image compression method based on multiple-tables arithmetic coding (MTAC) method to encode a gray-level imagef. First, the MTAC method employs a median edge detector (MED) to reduce the entropy rate off. The gray levels of two adjacent pixels in an image are usually similar. A base-switching transformation approach is then used to reduce the spatial redundancy of the image. The gray levels of some pixels in an image are more common than those of others. Finally, the arithmetic encoding method is applied to reduce the coding redundancy of the image. To promote high performance of the arithmetic encoding method, the MTAC method first classifies the data and then encodes each cluster of data using a distinct code table. The experimental results show that, in most cases, the MTAC method provides a higher efficiency in use of storage space than the lossless JPEG2000 does.
Universal lossless compression via multilevel pattern matching
IEEE Transactions on Information Theory, 2000
A universal lossless data compression code called the multilevel pattern matching code (MPM code) is introduced. In processing a finite-alphabet data string of length , the MPM code operates at (log log) levels sequentially. At each level, the MPM code detects matching patterns in the input data string (substrings of the data appearing in two or more nonoverlapping positions). The matching patterns detected at each level are of a fixed length which decreases by a constant factor from level to level, until this fixed length becomes one at the final level. The MPM code represents information about the matching patterns at each level as a string of tokens, with each token string encoded by an arithmetic encoder. From the concatenated encoded token strings, the decoder can reconstruct the data string via several rounds of parallel substitutions. A (1 log) maximal redundancy/sample upper bound is established for the MPM code with respect to any class of finite state sources of uniformly bounded complexity. We also show that the MPM code is of linear complexity in terms of time and space requirements. The results of some MPM code compression experiments are reported.
Lossless Image Compression
Digital Image processing has become ubiquitous in our daily life and the demands to produce and process images are ever increasing. Large amounts of space are required to store these images. Image compression techniques are in high demand as they allow reduction in this storage space. The basis for image compression, as is for most other compression techniques, is to remove redundant and unimportant data. Lossless image compression techniques retain the original information in compact form and do not introduce any errors when decompressed. In this paper, we discuss such a lossless technique using a data structure that we name "Generic Peano Pattern Mask Tree". It is an improvement over a previously discussed Lossless Image compression technique -"Peano Pattern Mask Tree" . Both these structures are based on the data structure -Peano Mask Tree.
An Efficient Technique of Lossless Data Compression
We present a new method for lossless image compression that gives compression comparable to JPEG lossless mode with about five times the speed. Our method, called ELICS, is based on a novel use of two neighboring pixels for both prediction and error modeling. For coding we use single bits, adjusted binary codes, and Golomb Rice codes. For the latter we present and analyze a provably good method for estimating the single coding parameter. Efficient, lossless image compression system (ELICS) algorithm, which consists of simplified adjusted binary code and Golomb–Rice code with storage-less k parameter selection, is proposed to provide the lossless compression method for high-throughput applications. The simplified adjusted binary code reduces the number of arithmetic operation and improves processing speed. According to theoretical analysis, the storage-less k parameter selection applies a fixed value in Golomb–Rice code to remove data dependency and extra storage for cumulation table.
Survey of Lossless Data Compression Algorithms
International Journal of Engineering Research and, 2015
The main goal of data compression is to decrease redundancy in warehouse or communicated data, so growing effective data density. It is a common necessary for most of the applications. Data compression is very important relevancy in the area of file storage and distributed system just because of in distributed system data have to send from and to all system. Two configuration of data compression are there "lossy" and "lossless". But in this paper we only focus on Lossless data compression techniques. In lossless data compression, the wholeness of data is preserved. Data compression is a technique that decreases the data size, removing the extreme information. Data compression has many types of techniques that decrease redundancy. The methods which mentioned are Run Length Encoding, Shannon Fanon, Huffman, Arithmetic, adaptive Huffman, LZ77, LZ78 and LZW with its performance.
A Lossless Data Compression and Decompression Algorithm and Its Hardware Architecture
IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2000
In this paper, we propose a new two-stage hardware architecture that combines the features of both parallel dictionary LZW (PDLZW) and an approximated adaptive Huffman (AH) algorithms. In this architecture, an ordered list instead of the treebased structure is used in the AH algorithm for speeding up the compression data rate. The resulting architecture shows that it not only outperforms the AH algorithm at the cost of only one-fourth the hardware resource but it is also competitive to the performance of LZW algorithm (compress). In addition, both compression and decompression rates of the proposed architecture are greater than those of the AH algorithm even in the case realized by software.
Technical note: New table look-up lossless compression method based on binary index archiving
International Journal of Engineering, Science and Technology, 2010
This paper intends to present a common use archiver, made up following the dictionary technique and using the index archiving method as a simple and original procedure. The original contribution of the paper consists in the structure of the archived file and in the transformation of the dictionary codes into archived characters. This archiver is useful in order to accomplish the lossless compression for any file types. The application can offer important conclusions regarding the compression performances and the influence of the chosen dictionary over the parameters.
Lossless Image Compression Using Data Folding Followed By Arithmetic Coding
The paper presents a lossless image compression technique using the hybridization of two different entropy coding techniques. Initially data folding technique has been applied to the image. A row folding is applied on the image matrix followed by a column folding. Multiple iterations of this process is applied on the image. After completing the data folding process another entropy coding technique known as arithmetic coding has been applied to the resultant image to get better results.
Analysis of Lossless Reversible Transformation Algorithms to Enhance Data Compression
Journal of Global Research in Computer Science, 2012
In this paper we analyze and present the benefits offered in the lossless compression by applying a choice of preprocessing methods that exploits the advantage of redundancy of the source file. Textual data holds a number of properties that can be taken int o account in order to improve compression. Pre-processing cope up with these properties by applying a number of transformations that make the redundancy "more visible" to the compressor. Many pre-processing algorithms come into being for text files which complement each other and are performed prior to actual compression. Here our focus is on the Length-Index Preserving Transform (LIPT), its derivatives ILPT, NIT & LIT and StarNT Transformation algorithm. The algorithms are briefly presented before calling attention to their analysis.
A Novel Lossless Encoding Algorithm for Data Compression
2021
Data compression is a challenging and increasingly important problem. As the amount of data generated daily continues to increase, efficient transmission and storage has never been more critical. In this study, a novel encoding algorithm is proposed, motivated by the compression of DNA data and associated characteristics. The proposed algorithm follows a divide-and-conquer approach by scanning the whole genome, classifying subsequences based on similarity patterns, and binning similar subsequences together. The data are then compressed in each bin independently. This approach is different than the currently known approaches: entropy, dictionary, predictive, or transform based methods. Proof-of-concept performance was evaluated using a benchmark dataset with seventeen genomes ranging in size from kilobytes to gigabytes. The results showed considerable improvement in the compression of each genome, preserving several megabytes compared with state-of-art tools. Moreover, the algorithm ...
A Research Paper on Lossless Data Compression Techniques
This research paper provides lossless data compression techniques and comparison between them. Data Compression is a process which reduces the size of data removing excessive information from it. It reduces the redundancy in data representation to decrease the storage required for that data and thus also reduces the communication cost by using the available bandwidth effectively. Data compression is important application in the area of file storage and distributed system. For different data formats like text, audio, video and image files there are different data compression techniques. Mainly there are two forms of data compression:-Lossy and Lossless. But in the lossless data compression, the integrity of data is to be preserved.
Pattern-Based Data Compression
MICAI 2004: Advances in Artificial Intelligence, 2004
Most modern lossless data compression techniques used today, are based in dictionaries. If some string of data being compressed matches a portion previously seen, then such string is included in the dictionary and its reference is included every time it appears. A possible generalization of this scheme is to consider not only strings made of consecutive symbols, but more general patterns with gaps between its symbols. The main problems with this approach are the complexity of pattern discovery algorithms and the complexity for the selection of a good subset of patterns. In this paper we address the last of these problems. We demonstrate that such problem is NP-complete and we provide some preliminary results about heuristics that points to its solution.
Data Compression Methodologies for Lossless Data and Comparison between Algorithms
This research paper provides lossless data compression methodologies and compares their performance. Huffman and arithmetic coding are compare according to their performances. Data compression is a process that reduces the data size, removing the excessive information. Shorter data size is suitable because it simply reduces the cost. The aim of data compression is to reduce redundancy in stored or communicated data, thus increasing effective data density. Data compression is important application in the area of file storage and distributed system because in distributed system data have to send from and to all system. So for speed and performance efficiency data compression is used. There are number of different data compression methodologies, which are used to compress different data formats like text, video, audio, image files. There are two forms of data compression “lossy” and “lossless”, in lossless data compression, the integrity of data is preserved.
Lossless data compression
2015
This thesis makes several contributions to the field of data compression. Lossless data compression algorithms shorten the description of input objects, such as sequences of text, in a way that allows perfect recovery of the original object. Such algorithms exploit the fact that input objects are not uniformly distributed: by allocating shorter descriptions to more probable objects and longer descriptions to less probable objects, the expected length of the compressed output can be made shorter than the object’s original description. Compression algorithms can be designed to match almost any given probability distribution over input objects. This thesis employs probabilistic modelling, Bayesian inference, and arithmetic coding to derive compression algorithms for a variety of applications, making the underlying probability distributions explicit throughout. A general compression toolbox is described, consisting of practical algorithms for compressing data distributed by various fund...
A Memory-Assisted Lossless Compression using Combination Methods
International Journal of Computer Applications, 2015
Image compression can be done in two ways namely lossy or lossless. Lossy methods are used for expected results like photographs where some negligible loss of data is tolerable. Rapid growth of medical science such as ehealth and telemedicine requires lossless Image compression. The correlation and redundancy which exists across different medical images are considered to achieve better compression ratio.. Better compression results in less storage and less overhead while transmission on the network. A memoryassisted hybrid compression technique is proposed. The approach is motivated by using a combination of methods, lossy and lossless. The original image is compressed using PCA algorithm and then using lossless coding methods. PCA is used to find the correlation among the similar medical images. PCA algorithm is used with only some principal components.
Data Compression-Lossless and Lossy Techniques
2016
Compression reduces the number of bits required to represent the data. Compression is useful because it helps in reducing the consumption of expensive resources, such as disk space and transmission bandwidth. Compression is built into a broad range of technologies like storage systems, databases operating systems and software applications. Hence selection of data compression algorithm should be appropriate. This paper presents different data compression methodologies. Mainly there are two forms of data compression :Lossless and Lossy. In this paper, we discussed about some of the Lossless and Lossy data compression methods.
Lossless and Nearly-Lossless Image Compression Based on Combinatorial Transforms
2011
Common image compression standards are usually based on frequency transform such as Discrete Cosine Transform or Wavelets. We present a different approach for lossless image compression, it is based on combinatorial transform. The main transform is Burrows Wheeler Transform (BWT) which tends to reorder symbols according to their following context. It becomes a promising compression approach based on context modelling. BWT was initially applied for text compression software such as BZIP2; nevertheless it has been recently applied to the image compression field. Compression scheme based on Burrows Wheeler Transform is usually lossless; therefore we implement this algorithm in medical imaging in order to reconstruct every bit. Many variants of the three stages which form the original BWT-based compression scheme can be found in the literature. We propose an analysis of the more recent methods and the impact of their association. Then, we present several compression schemes based on this transform which significantly improve the current standards such as JPEG2000 and JPEG-LS. In the final part, we present some open problems which are also further research directions.
Arithmetic coding for lossless and loss-inducing image compression
Sixth Multidimensional Signal Processing Workshop
Arithmetic coding is applied to provide lossless and loss-inducing compression of optical, infrared, and synthetic aperture radar imagery of natural scenes. Arithmetic coding algorithms successfully exploit the dependence structure of images through the adaptive estimation of probability distributions conditioned on pixel contexts. Several different contexts are considered, including both predictive and non-predictive variations, with both image-dependent and image-independent variations. In lossless coding experiments, arithmetic coding algorithms are shown to outperform comparable variants of both Huffman and Lempel-Ziv-Welch coding algorithms by approximately 0.5 bits per pixel. For image-dependent contexts constructed from high-order autoregressive predictors, arithmetic coding algorithms provide compression ratios as high as 4. Contexts constructed from lower-order auteregressive predictors provide compression ratios nearly as great as those of the higher-order predictors with favorable computational trades. Compression performance variations are shown to reflect the inherent sensor-dependent differences in the stochastic structure of the imagery. Arithmetic coding is also demonstrated to be a valuable addition to loss-inducing compression techniques. Code sequences derived from a lapped orthogonal transform-based vector quantization scheme are shown to be losslessly compressible using the arithmetic coding scheme. For imagery compressed to 0.5 bits per pixel, the addition of an arithmetic coder with Markov-dependent context results in additional compression ratio gains as high as 2 with no additional loss in fidelity. 'Rome Air Development Center sponsored this work under contract number F30602-87-(3-0225.
Data Compression Using a Dictionary of Patterns
2000
Most modern lossless data compression techniques used to- day, are based in dictionaries. If some string of data being compressed matches a portion previously seen, then such string is included in the dictionary and its reference is included every time it occurs. A possi- ble generalization of this scheme is to consider not only strings made of consecutive symbols, but
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.