On the Application of Kolmogorov Complexity to

The Computer Journal, 1999

The question why and how probability theory can be applied to the real-world phenomena has been discussed for several centuries. When the algorithmic information theory was created, it became possible to discuss these problems in a more specific way. In particular, Li and Vitányi [6], Rissanen [3], Wallace and Dowe [7] have discussed the connection between Kolmogorov (algorithmic) complexity and minimum description length (minimum message length) principle. In this note we try to point out a few simple observations that (we believe) are worth keeping in mind while discussing these topics.

This paper introduces a new methodology, based on Kolmogorov complexity, to compare and evaluate computer animation models, regarding data compression. It was originally proposed to be applyed to characterize the software development process. The methodology proposed here may be used on a more general context, as a tool to solve problems in systems science. Systems science has a vague definition. It is related to the study of general principles governing systems of a large range of types. In our method, we define an animation model as a machine and use machine simulation to compare diferent types of models, setting out an animation model hierarchy. A definition of data compression is provided to support our definition of "better animation". The definition includes lossy data compression and may be applyed to MPEG streams. The methodology is theoretical and related to classic computing theory.

Journal of Information and Organizational Sciences

We present the modeling of dynamical systems and …nding of their complexity indicators by the use of concepts from computation and information theories, within the framework of J. P. Crutch…eld's theory of-machines. A short formal outline of the-machines is given. In this approach, dynamical systems are analyzed directly from the time series that is received from a properly adjusted measuring instrument. The paper serves also as a theoretical foundation for the future presentation of the DSA program that implements the-machines modeling up to the stochastic …nite automata level.

International Journal of Engineering Sciences & Research Technology, 2012

This paper describes various application issues of kolmogorov complexity . Information assurance , network management , active network are those areas where kolmogorov complexity is applied . Our main focus is to show it's importance in various domain including the domain of computer virus detection .

Chaos, Solitons & Fractals, 1994

A number of different measures of complexity have been described, discussed, and applied to the logistic map. A classification of these measures has been proposed, distinguishing homogeneous and generating partitions in phase space as well as structural and dynamical elements of the considered measure. The specific capabilities of particular measures to detect particular types of behavior of dynamical systems have been investigated and compared with each other. 134 R. WACKERBAUER el al.

The generalized Statistical Complexity Measure (SCM) is a functional that characterizes the probability distribution P associated to the time series generated by a dynamical system under study. It quantifies not only randomness but also the presence of correlational structures. In this seminar several fundamental issues are reviewed: a) selection of the information measure I; b) selection of the probability metric space and its corresponding distance D; c) definition of the generalized disequilibrium Q; d) selection of the probability distribution P associated to a dynamical system or time series under study, which in fact, is a basic problem. Here we show that improvements can be expected if the underlying probability distribution is "extracted" by appropriate consideration regarding causal effects in the system's dynamics. Several well-known model-generated time series, usually regarded as being of either stochastic or chaotic nature, are analyzed. The main achievement of this approach is the possibility of clearly distinguish between them in the Entropy-Complexity representation space, something that is rather difficult otherwise.

This paper presents a PhD research project focused on investigating the use of Kolmogorov complexity approximations as descriptors for various data types, with the aim of addressing inversion problems. The research explores the application of these approximations across different domains while considering the relationship between algorithmic and probabilistic complexities. The study starts with genomic data analysis, where specialized data compressors are employed to improve taxonomic identification, classification, and organization. The research then extends to analysing artistic paintings, utilizing information-based measures to attribute authorship, categorize styles, and describe the content. Additionally, the research examines Turing Machine-generated data, providing insights into the relationship between algorithmic and probabilistic complexities. A method for increasing probabilistic complexity without affecting algorithmic complexity is also proposed. Lastly, a methodology for identifying programs capable of generating outputs approximating given input strings is introduced, offering potential solutions to inversion problems. The paper highlights the diverse applications and findings from this research, contributing to understanding the relationship between algorithmic and probabilistic complexities in data analysis.

2010

This is a short introduction to Kolmogorov complexity and information theory. The interested reader is referred to the literature, especially the textbooks [CT91] and [LV97] which cover the fields of information theory and Kolmogorov complexity in depth and with all the necessary rigor. They are well to read and require only a minimum of prior knowledge.

Arxiv preprint nlin/0307013, 2003

Some aspects of the predictability problem in dynamical systems are reviewed. The deep relation among Lyapunov exponents, Kolmogorov-Sinai entropy, Shannon entropy and algorithmic complexity is discussed. In particular, we emphasize how a characterization of the unpredictability of a system gives a measure of its complexity. A special attention is devoted to finite-resolution effects on predictability, which can be accounted with suitable generalization of the standard indicators. The problems involved in systems with intrinsic randomness is discussed, with emphasis on the important problems of distinguishing chaos from noise and of modeling the system. PACS numbers: PACS 45.05.+x, 05.45.-a All the simple systems are simple in the same way, each complex system has its own complexity (freely inspired by Anna Karenina by Lev N.

2020

Complexity is a multi-faceted phenomenon, involving a variety of features including disorder, nonlinearity, and self-organisation. We use a recently developed rigorous framework for complexity to understand measures of complexity. We illustrate, by example, how features of complexity can be quantified, and we analyse a selection of purported measures of complexity that have found wide application and explain whether and how they measure complexity. We also discuss some of the classic information-theoretic measures from the 1980s and 1990s. This work gives the reader a tool kit for quantifying features of complexity across the sciences.

PLoS ONE, 2010

Background: The evaluation of the complexity of an observed object is an old but outstanding problem. In this paper we are tying on this problem introducing a measure called statistic complexity.

2001

Abstract The problem of information assurance is approached from the point of view of Kolmogorov complexity and minimum message length criteria. Several theoretical results are obtained, possible applications are discussed and a new metric for measuring complexity is introduced. Utilization of Kolmogorov complexity like metrics as conserved parameters to detect abnormal system behavior is explored.

Kolmogorov complexity furnishes many useful tools for studying different natural processes that can be expressed using sequences of symbols from a finite alphabet (texts), such as genetic texts, literary and music texts, animal communications, etc. Although Kolmogorov complexity is not algorithmically computable, in a certain sense it can be estimated by means of data compressors. Here we suggest a method of analysis of sequences based on ideas of Kolmogorov complexity and mathematical statistics, and apply this method to biological (ethological) "texts." A distinction of the suggested method from other approaches to the analysis of sequential data by means of Kolmogorov complexity is that it belongs to the framework of mathematical statistics, more specifically, that of hypothesis testing. This makes it a promising candidate for being included in the toolbox of standard biological methods of analysis of different natural texts, from DNA sequences to animal behavioural patterns (ethological "texts"). Two examples of analysis of ethological texts are considered in this paper. Theses examples show that the proposed method is a useful tool for Theory Comput Syst distinguishing between stereotyped and flexible behaviours, which is important for behavioural and evolutionary studies.

Entropy, 2013

Permutation entropy, introduced by Bandt and Pompe, is a conceptually simple and well-interpretable measure of time series complexity. In this paper, we propose efficient methods for computing it and related ordinal-patterns-based characteristics. The methods are based on precomputing values of successive ordinal patterns of order d, considering the fact that they are "overlapped" in d points, and on precomputing successive values of the permutation entropy related to "overlapping" successive time-windows. The proposed methods allow for measurement of the complexity of very large datasets in real-time.

Chaos, Solitons & Fractals, 2003

The adoption of the Kolmogorov-Sinai (KS) entropy is becoming a popular research tool among physicists, especially when applied to a dynamical system fitting the conditions of validity of the Pesin theorem. The study of time series that are a manifestation of system dynamics whose rules are either unknown or too complex for a mathematical treatment, is still a challenge since the KS entropy is not computable, in general, in that case. Here we present a plan of action based on the joint action of two procedures, both related to the KS entropy, but compatible with computer implementation through fast and efficient programs. The former procedure, called Compression Algorithm Sensitive To Regularity (CASToRe), establishes the amount of order by the numerical evaluation of algorithmic compressibility. The latter, called Complex Analysis of Sequences via Scaling AND Randomness Assessment (CASSANDRA), establishes the complexity degree through the numerical evaluation of the strength of an anomalous effect. This is the departure, of the diffusion process generated by the observed fluctuations, from ordinary Brownian motion. The CASSANDRA algorithm shares with CASToRe a connection with the Kolmogorov complexity. This makes both algorithms especially suitable to study the transition from dynamics to thermodynamics, and 1 the case of non-stationary time series as well. The benefit of the joint action of these two methods is proven by the analysis of artificial sequences with the same main properties as the real time series to which the joint use of these two methods will be applied in future research work.