Algorithmic Information Theory

While Kolmogorov complexity is the accepted absolute measure of information content in an individual finite object, a similarly absolute notion is needed for the information distance between two individual objects, for example, two pictures. We give several natural definitions of a universal information metric, based on length of shortest programs for either ordinary computations or reversible (dissipationless) computations. It turns out that these definitions are equivalent up to an additive logarithmic term. We show that the information distance is a universal cognitive similarity distance. We investigate the maximal correlation of the shortest programs involved, the maximal uncorrelation of programs (a generalization of the Slepian-Wolf theorem of classical information theory), and the density properties of the discrete metric spaces induced by the information distances. A related distance measures the amount of nonreversibility of a computation. Using the physical theory of reversible computation, we give an appropriate (universal, anti-symmetric, and transitive) measure of the thermodynamic work required to transform one object in another object by the most efficient process. Information distance between individual objects is needed in pattern recognition where one wants to express effective notions of "pattern similarity" or "cognitive similarity" between individual objects and in thermodynamics of computation where one wants to analyse the energy dissipation of a computation from a particular input to a particular output.

As an increasing number of protein structures become available, the need for algorithms that can quantify the similarity between protein structures increases as well. Thus, the comparison of proteins' structures, and their clustering accordingly to a given similarity measure, is at the core of today's biomedical research. In this paper, we show how an algorithmic information theory inspired Universal Similarity Metric (USM) can be used to calculate similarities between protein pairs. The method, besides being theoretically supported, is surprisingly simple to implement and computationally efficient. Results: Structural similarity between proteins in four different datasets was measured using the USM.The sample employed represented alpha, beta, alpha-beta, tim-barrel, globins and serpine protein types. The use of the proposed metric allows for a correct measurement of similarity and classification of the proteins in the four datasets. Availability: All the scripts and programs used for the preparation of this paper are available at http://www.cs.nott.ac.uk/ nxk/USM/protocol.html. In that web-page the reader will find a brief description on how to use the various scripts and programs. Contact: Natalio.Krasnogor@nottingham.ac.uk; dpelta@ugr.es Supplementary information: The protein datasets used are collected in

The algorithmic theory of randomness is well developed when the underlying space is the set of finite or infinite sequences and the underlying probability distribution is the uniform distribution or a computable distribution. These restrictions seem artificial. Some progress has been made to extend the theory to arbitrary Bernoulli distributions (by Martin-Löf), and to arbitrary distributions (by Levin). We recall the main ideas and problems of Levin's theory, and report further progress in the same framework. The issues are the following: -Allow non-compact spaces (like the space of continuous functions, underlying the Brownian motion). -The uniform test (deficiency of randomness) dP (x) (depending both on the outcome x and the measure P ) should be defined in a general and natural way. -See which of the old results survive: existence of universal tests, conservation of randomness, expression of tests in terms of description complexity, existence of a universal measure, expression of mutual information as "deficiency of independence". -The negative of the new randomness test is shown to be a generalization of complexity in continuous spaces; we show that the addition theorem survives. The paper's main contribution is introducing an appropriate framework for studying these questions and related ones (like statistics for a general family of distributions).

In the study of nonlinear physical systems, one encounters apparently random or chaotic behavior, although the systems may be completely deterministic. Applying techniques from symbolic dynamics to maps of the interval, we compute two measures of chaotic behavior commonly employed in dynamical systems theory: the topological and metric entropies. For the quadratic logistic equation, we find that the metric entropy converges very slowly in comparison to maps which are strictly hyperbolic. The effects of finite precision arithmetic and external noise on chaotic behavior are characterized with the symbolic dynamics entropies. Finally, we discuss the relationship of these measures of chaos to algorithmic complexity, and use algorithmic information theory as a framework to discuss the construction of models for chaotic dynamics.

Effective fractal dimension was defined by Lutz (2003) in order to quantitatively analyze the structure of complexity classes, but then interesting connections of effective dimension with information theory were also found, justifying the long existent intuition that dimension is an information content measure. Considering different bounds on computing power that range from finite memory to constructibility, including time-bounded and space-bounded

An algorithmic information theoretic method is presented for object-level summarization of meaningful changes in image sequences. Object extraction and tracking data are represented as an attributed tracking graph (ATG), whose connected subgraphs are compared using an adaptive information distance measure, aided by a closed-form multi-dimensional quantization. The summary is the clustering result and feature subset that maximize the gap statistic. The notion of meaningful summarization is captured by using the gap statistic to estimate the randomness deficiency from algorithmic statistics. When applied to movies of cultured neural progenitor cells, it correctly distinguished neurons from progenitors without requiring the use of a fixative stain. When analyzing intra-cellular molecular transport in cultured neurons undergoing axon specification, it automatically confirmed the role of kinesins in axon specification. Finally, it was able to differentiate wild type from genetically modified thymocyte cells.

In contrast with software-generated randomness (called pseudo-randomness), quantum randomness can be proven incomputable; that is, it is not exactly reproducible by any algorithm. We provide experimental evidence of incomputability-an asymptotic property-of quantum randomness by performing finite tests of randomness inspired by algorithmic information theory.

Four general approaches to the metaphysics of causation are current in
Australasian philosophy. One is a development of the regularity theory (attributed to Hume) that uses counterfactuals (Lewis, 1973; 1994). A second is based in the relations of universals, which determine laws, which in turn determine causal interactions of particulars (with the possible exception of singular causation, Armstrong, 1983). This broad approach goes back to Plato, and was also held in this century by Russell, who like Plato, but unlike the more recent version of Armstrong (1983), held there were no particulars as such, only universals. A third view, originating with Reichenbach and revived by Salmon (1984), holds that a causal process is one that can be marked. This view relies heavily on ideas about the transfer of information and the relation of information to probability, but it also needs uneliminable counterfactuals. The fourth view was developed recently by Dowe (1992) and Salmon (1994). It holds that a causal process involves the transfer of a non-zero valued conserved quantity. A considerable advantage of this approach over the others is that it requires neither counterfactuals nor abstracta like universals to explain causation.
The theory of causation offered here is a development of the mark approach that entails Dowe’s conserved quantity approach. The basic idea is that causation is the transfer of a particular token of a quantity of information from one state of a system to another. Physical causation is a special case in
which physical information instances are transferred from one state of a physical system to another. The approach can be interpreted as a Universals approach (depending on ones approach to mathematical objects and qualities), and it sheds some light on the nature of the regularity approach.
After motivating and describing this approach, I will sketch how it can be used to ground natural laws and how it relates to the four leading approaches, in particular how each can be conceived as a special case of my approach. Finally, I will show how my approach satisfies the requirements of Humean supervenience. The approach relies on concrete particulars and computational logic alone, and is the second stage of constructing a minimal metaphysics, started in (Collier 1996, The necessity of natural kinds).

This article investigates emergence and complexity in complex systems that can share information on a network. To this end, we use a theoretical approach from information theory, computability theory, and complex networks. One key studied question is how much emergent complexity arises when a population of computable systems is networked compared with when this population is isolated. First, we define a general model for networked theoretical machines, which we call algorithmic networks. Then, we narrow our scope to investigate algorithmic networks that optimize the average fitnesses of nodes in a scenario in which each node imitates the fittest neighbor and the randomly generated population is networked by a time-varying graph. We show that there are graph-topological conditions that cause these algorithmic networks to have the property of expected emergent open-endedness for large enough populations. In other words, the expected emergent algorithmic complexity of a node tends to infinity as the population size tends to infinity. Given a dynamic network, we show that these conditions imply the existence of a central time to trigger expected emergent open-endedness. Moreover, we show that networks with small diameter compared to the network size meet these conditions. We also discuss future research based on how our results are related to some problems in network science, information theory, computability theory, distributed computing, game theory, evolutionary biology, and synergy in complex systems.

Deep learning and other similar machine learning techniques have a huge advantage over other AI methods: they do function when applied to real-world data, ideally from scratch, without human intervention. However, they have several shortcomings that mere quantitative progress is unlikely to overcome. The paper analyses these shortcomings as resulting from the type of compression achieved by these techniques, which is limited to statistical compression. Two directions for qualitative improvement, inspired by comparison with cognitive processes, are proposed here, in the form of two mechanisms: complexity drop and contrast. These mechanisms are supposed to operate dynamically and not through pre-processing as in neural networks. Their introduction may bring the functioning of AI away from mere reflex and closer to reflection.

In spite of being ubiquitous in life sciences, the concept of information is harshly criticized. Uses of the concept other than those derived from Shannon's theory are denounced as pernicious metaphors. We perform a computational experiment to explore whether Shannon's information is adequate to describe the uses of said concept in commonplace scientific practice. Our results show that semantic sequences do not have unique complexity values different from the value of meaningless sequences. This result suggests that quantitative theoretical frameworks do not account fully for the complex phenomenon that the term “information” refers to. We propose a restructuring of the concept into two related, but independent notions, and conclude that a complete theory of biological information must account completely not only for both notions, but also for the relationship between them.

This document contains lecture notes of an introductory course on Kolmogorov complexity. They cover basic notions of algorithmic information theory: Kolmogorov complexity (plain, conditional, prefix), notion of randomness (Martin-Löf randomness, Mises–Church randomness), Solomonoff universal a priori probability and their properties (symmetry of information, connection between a priori probability and prefix complexity, criterion of randomness in terms of complexity) and applications (incompressibility method in computational complexity theory, incompleteness theorems).

This paper is devoted to large scale aspects of the geometry of the space of isometry classes of Riemannian metrics, with a 2-sided curvature bound, on a fixed compact smooth manifold of dimension at least five. Using a mix of tools from logic/computer science, and differential geometry and topology, we study the diameter functional and its critical points, as well as their distribution (density) within the space and the structure of their neighborhoods.

The concept of entropy plays a major part in communication theory. The Shannon entropy is a measure of uncertainty with respect to a priori probability distribution. In algorithmic information theory the information content of a message is measured in terms of the size in bits of the smallest program for computing that message. This paper discusses the classical entropy and entropy rate for discrete or continuous Markov sources, with finite or continuous alphabets, and their relations to program-size complexity and algorithmic probability. The accent is on ideas, constructions and results; no proofs will be given.

Using frequency distributions of daily closing price time series of several financial market indexes, we investigate whether the bias away from an equiprobable sequence distribution found in the data, predicted by algorithmic information theory, may account for some of the deviation of financial markets from log-normal, and if so for how much of said deviation and over what sequence lengths. We do so by comparing the distributions of binary sequences from actual time series of financial markets and series built up from purely algorithmic means. Our discussion is a starting point for a further investigation of the market as a rule-based system with an algorithmic component, despite its apparent randomness, and the use of the theory of algorithmic probability with new tools that can be applied to the study of the market price phenomenon. The main discussion is cast in terms of assumptions common to areas of economics in agreement with an algorithmic view of the market.

In a sampling problem, we are given an input x ∈ {0, 1} n , and asked to sample approximately from a probability distribution D x over poly (n)-bit strings. In a search problem, we are given an input x ∈ {0, 1} n , and asked to find a member of a nonempty set A x with high probability. (An example is finding a Nash equilibrium.) In this paper, we use tools from Kolmogorov complexity and algorithmic information theory to show that sampling and search problems are essentially equivalent. More precisely, for any sampling problem S, there exists a search problem R S such that, if C is any "reasonable" complexity class, then R S is in the search version of C if and only if S is in the sampling version.

"Abstract. Ray Solomonoff invented the notion of universal induction
featuring an aptly termed “universal” prior probability function over all
possible computable environments [9]. The essential property of this prior
was its ability to dominate all other such priors. Later, Levin introduced
another construction — a mixture of all possible priors or “universal
mixture” [12]. These priors are well known to be equivalent up to mul-
tiplicative constants. Here, we seek to clarify further the relationships
between these three characterisations of a universal prior (Solomonoff’s,
universal mixtures, and universally dominant priors). We see that the
the constructions of Solomonoff and Levin define an identical class of
priors, while the class of universally dominant priors is strictly larger.
We provide some characterisation of the discrepancy.
"

Kolmogorov's very first paper on algorithmic information theory (Kolmogorov, Problemy peredachi infotmatsii 1(1) (1965), 3) was entitled "Three approaches to the definition of the quantity of information". These three approaches were called combinatorial, probabilistic and algorithmic. Trying to establish formal connections between combinatorial and algorithmic approaches, we prove that every linear inequality including Kolmogorov complexities could be translated into an equivalent combinatorial statement. (Note that the same linear inequalities are true for Kolmogorov complexities and Shannon entropy, see Hammer et al., (Proceedings of CCC'97, Ulm).) Entropy (complexity) proofs of combinatorial inequalities given in Llewellyn and Radhakrishnan (Personal Communication) and Hammer and Shen (Theory Comput. Syst. 31 (1998) 1) can be considered as special cases (and a natural starting points) for this translation.

We further deconstruct Heraclitean Quantum Systems giving a model for a universe using pregeometric notions in which the end-game problem is overcome by means of self-referential noise. The model displays self-organisation with the emergence of 3-space and time. The time phenomenon is richer than the present geometric modelling.

The main topic of the present work are universal machines for plain and prefix-free description complexity and their domains. It is characterised when an r.e. set W is the domain of a universal plain machine in terms of the description complexity of the spectrum function s W mapping each non-negative integer n to the number of all strings of length n in W ; furthermore, a characterisation of the same style is given for supersets of domains of universal plain machines. Similarly the prefix-free sets which are domains or supersets of domains of universal prefix-free machines are characterised. Furthermore, it is shown that the halting probability Ω V of an r.e. prefix-free set V containing the domain of a universal prefix-free machine is Martin-Löf random, while V may not be the domain of any universal prefix-free machine itself. Based on these investigations, the question whether every domain of a universal plain machine is the superset of the domain of some universal prefix-free machine is discussed. A negative answer to this question had been presented at CiE 2010 by Mikhail Andreev, Ilya Razenshteyn and Alexander Shen, while this paper was under review.

An algorithmic information theoretic method is presented for object-level summarization of meaningful changes in image sequences. Object extraction and tracking data are represented as an attributed tracking graph (ATG), whose connected subgraphs are compared using an adaptive information distance measure, aided by a closed-form multi-dimensional quantization. The summary is the clustering result and feature subset that maximize the gap statistic. The notion of meaningful summarization is captured by using the gap statistic to estimate the randomness deficiency from algorithmic statistics. When applied to movies of cultured neural progenitor cells, it correctly distinguished neurons from progenitors without requiring the use of a fixative stain. When analyzing intra-cellular molecular transport in cultured neurons undergoing axon specification, it automatically confirmed the role of kinesins in axon specification. Finally, it was able to differentiate wild type from genetically modified thymocyte cells.

We discuss certain analogies between quantization and discretization of classical systems on manifolds. In particular, we will apply the quantum dynamical entropy of Alicki and Fannes to numerically study the footprints of chaos in discretized versions of hyperbolic maps on the torus.

We show that Kolmogorov complexity and such its estimators as universal codes (or data compression methods) can be applied for hypothesis testing in a framework of classical mathematical statistics. The methods for identity testing and nonparametric testing of serial independence for time series are described.

Recent large scale experiments have shown that the Normalized Information Distance, an algorithmic information measure, is among the best similarity metrics for melody classification. This paper proposes the use of this distance as a fitness function which may be used by genetic algorithms to automatically generate music in a given pre-defined style. The minimization of this distance of the generated music to a set of musical guides makes it possible to obtain computer-generated music which recalls the style of a certain human author. The recombination operator plays an important role in this problem and thus several variations are tested to fine tune the genetic algorithm for this application. The superiority of the relative pitch envelope over other music parameters, such as the lengths of the notes, brought us to develop a simplified algorithm that nevertheless obtains interesting results.

Recently, genetic programming has been proposed to model agents' adaptive behavior in a complex transition process where uncertainty cannot be formalized within the usual probabilistic framework. However, this approach has not been widely accepted by economists. One of the main reasons is the lack of the theoretical foundation of using genetic programming to model transition dynamics. Therefore, the purpose of this paper is two-fold. First, motivated by the recent applications of algorithmic information theory in economics, we would like to show the relevance of genetic programming to transition dynamics given this background. Second, we would like to supply two concrete applications to transition dynamics. The first application, which is designed for the pedagogic purpose, shows that genetic programming can simulate the non-smooth transition, which is difficult to be captured by conventional toolkits, such as differential equations and difference equations. In the second application, genetic programming is applied to simulate the adaptive behavior of speculators. This simulation shows that genetic programming can generate artificial time series with the statistical properties frequently observed in real financial time series.

The main contribution of this paper is to design an Information Retrieval (IR) technique based on Algorithmic Information Theory (using the Normalized Compression Distance-NCD), statistical techniques (outliers), and novel organization of data base structure. The paper shows how they can be integrated to retrieve information from generic databases using long (text-based) queries. Two important problems are analyzed in the paper. On the one hand, how to detect "false positives" when the distance among the documents is very low and there is actual similarity. On the other hand, we propose a way to structure a document database which similarities distance estimation depends on the length of the selected text. Finally, the experimental evaluations that have been carried out to study previous problems are shown.

The concept of effective complexity of an object as the minimal description length of its regularities has been initiated by Gell-Mann and Lloyd. The regularities are modeled by means of ensembles, which is the probability distributions on finite binary strings. In our previous paper [1] we propose a definition of effective complexity in precise terms of algorithmic information theory. Here we investigate the effective complexity of binary strings generated by stationary, in general not computable, processes. We show that under not too strong conditions long typical process realizations are effectively simple. Our results become most transparent in the context of coarse effective complexity which is a modification of the original notion of effective complexity that needs less parameters in its definition. A similar modification of the related concept of sophistication has been suggested by Antunes and Fortnow.

Although information content is invariant up to an additive constant, the range of possible additive constants applicable to programming languages is so large that in practice it plays a major role in the actual evaluation of K(s), the Kolmogorov-Chaitin complexity of a string s. Some attempts have been made to arrive at a framework stable enough for a concrete definition of K, independent of any constant under a programming language, by appealing to the naturalness of the language in question. The aim of this paper is to present an approach to overcome the problem by looking at a set of models of computation converging in output probability distribution such that that naturalness can be inferred, thereby providing a framework for a stable definition of K under the set of convergent models of computation.

In a genetic algorithm, fluctuations of the entropy of a genome over time are interpreted as fluctuations of the information that the genome's organism is storing about its environment, being this reflected in more complex organisms. The computation of this entropy presents technical problems due to the small population sizes used in practice. In this work we propose and test an alternative way of measuring the entropy variation in a population by means of algorithmic information theory, where the entropy variation between two generational steps is the Kolmogorov complexity of the first step conditioned to the second one. As an example application of this technique, we report experimental differences in entropy evolution between systems in which sexual reproduction is present or absent.

In  this  article,  we  will  show  that  uncomputability  is  a  relative property  not  only  of  oracle  Turing  machines,  but  also  of  subrecursive classes. We will define the concept of a Turing submachine, and a recursive relative  version  for  the  Busy Beaver  function which we  will  call  Busy Beaver Plus function. Therefore, we will prove that the computable Busy Beaver Plus function defined on any Turing submachine is not computable by any program running on this submachine. We will thereby demonstrate the existence of a “paradox” of computability a la Skolem: a function is computable when “seen from the outside” the subsystem, but uncomputable when “seen from within” the same subsystem. Finally, we will raise the possibility of defining universal submachines,  and a hierarchy of negative Turing degrees.

Let v, w be infinite 0-1 sequences, andm a positive integer. We say that w ism-embeddable in v, if there exists an increasing sequence (n i : i ≥ 0) of integers with n 0 = 0, such that 1 ≤ n i − n i −1 ≤m, w(i) = v(n i) for all i ≥ 1. Let X and Y be coin-tossing sequences. We will show that there is anm with the property that Y ism-embeddable into X with positive probability. This answers a question that was open for a while. The proof generalizes somewhat the hierarchical method of an earlier paper of the author on dependent percolation.

The sophistication of a string measures how much structural information it contains. We introduce naive sophistication, a variant of sophistication based on randomness deficiency. Naive sophistication measures the minimum number of bits needed to specify a set in which the string is a typical element. Thanks to Vereshchagin and Vitányi, we know that sophistication and naive sophistication are equivalent up to low order terms. We use this to relate sophistication to lossy compression, and to derive an alternative formulation for busy beaver computational depth.

Generalized Information (GI) is a measurement of the degree to which a program can be said to generalize a dataset. It is calculated by creating a program to model the data set, measuring the Active Information in the model, and subtracting out the size of the model. Active Information allows GI to be usable with both exact and inexact models.

This work presents a theoretical investigation of incompressible multidimensional networks defined by a generalized graph representation. In particular, we study the incompressibility (i.e., algorithmic randomness) of snapshot-dynamic networks and multiplex networks in comparison to the incompressibility of more general forms of multidimensional networks, from which snapshot-dynamic networks or multiplex networks are particular cases. In addition, we study some of their network topological properties and discuss how these may be related to real-world complex networks. First, we show that incompressible snapshot-dynamic (or multiplex) networks carry an amount of algorithmic information that is linearly dominated by the size of the set of time instants (or layers). This contrasts with the algorithmic information carried by an incompressible general dynamic (or multilayer) network that is of the quadratic order of the size of the set of time instants (or layers). Furthermore, incompressible general multidimensional networks inherit most of the topological properties from their respective isomorphic graph. Hence, we show that these networks have very short diameter, high k-connectivity, and degrees of the order of half of the network size within a strong-asymptotically dominated standard deviation. Particularly, we show that incompressible general multidimensional networks have transtemporal (or crosslayer) edges. Thus, this property may not correspond to the underlying structure of many real-world networks that can be properly modeled by snapshot-like multidimensional networks.

Broadly speaking, there are two approaches to quantifying information. The first, Shannon information, takes events as belonging to ensembles and quantifies the information resulting from observing the given event in terms of the number of alternate events that have been ruled out. The second, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as the length of the shortest program producing it. This note describes a new method of quantifying information, effective information, that links algorithmic information to Shannon information, and also links both to capacities arising in statistical learning theory. After introducing the measure, we show that it provides a non-universal analog of algorithmic information. We then apply it to derive basic capacities in statistical learning theory: empirical VC-entropy and empirical Rademacher complexity. A nice byproduct of our approach is an interpretation of the explanatory power of a learning algorithm in terms of the number of hypotheses it falsifies (counted in two different ways for the two different capacities). We also discuss how effective information relates to information gain, Shannon and mutual information.

Network complexity, network information content analysis, and lossless compressibility of graph representations have been played an important role in network analysis and network modeling. As multidimensional networks, such as time-varying, multilayer, or dynamic multilayer networks, gain more relevancy in network science, it becomes crucial to investigate in which situations universal algorithmic methods based on algorithmic information theory applied to graphs cannot be straightforwardly imported into the multidimensional case. In this direction, as a worst-case scenario of lossless compressibility distortion that increases linearly with the number of distinct dimensions, this article presents a counter-intuitive phenomenon that occurs when dealing with networks within non-uniform and sufficiently large multidimensional spaces. In particular, we demonstrate that the algorithmic information necessary to encode multidimensional networks that are isomorphic to logarithmically compressible monoplex networks may display exponentially larger distortions in the general case.

This article presents a theoretical investigation of computation beyond the Turing barrier from emergent behavior in distributed systems. In particular, we present an algorithmic network that is a mathematical model of a networked population of randomly generated computable systems with a fixed communication protocol. Then, in order to solve an undecidable problem, we study how nodes (i.e., Turing machines or computable systems) can harness the power of the metabiological selection and the power of information sharing (i.e., communication) through the network. Formally, we show that there is a pervasive network topological condition, in particular, the small-diameter phenomenon, that ensures that every node becomes capable of solving the halting problem for every program with a length upper bounded by a logarithmic order of the population size. In addition, we show that this result implies the existence of a central node capable of emergently solving the halting problem in the minimum number of communication rounds. Furthermore, we introduce an algorithmic-informational measure of synergy for networked computable systems, which we call local algorithmic synergy. Then, we show that such algorithmic network can produce an arbitrarily large value of expected local algorithmic synergy.

Our aim is to experimentally study the possibility of distinguishing between quantum sources of randomness-recently proved to be theoretically incomputable-and some well-known computable sources of pseudo-randomness. Incomputability is a necessary, but not sufficient "symptom" of "true randomness."

Solomonoff induction is known to be universal, but incomputable. Its approximations, namely, the Minimum Description (or Message) Length (MDL) principles, are adopted in practice in the efficient, but non-universal form. Recent attempts to bridge this gap leaded to development of the Representational MDL principle that originates from formal decomposition of the task of induction. In this paper, possible extension of the RMDL principle in the context of universal intelligence agents is considered, for which introduction of representations is shown to be an unavoidable meta-heuristic and a step toward efficient general intelligence. Hierarchical representations and model optimization with the use of information-theoretic interpretation of the adaptive resonance are also discussed.

In 1975, Chaitin introduced his celebrated Omega number, the halting probability of a universal Chaitin machine, a universal Turing machine with a prefix-free domain. The Omega number's bits are algorithmically random-there is no reason the bits should be the way they are, if we define "reason" to be a computable explanation smaller than the data itself. Since that time, only two explicit universal Chaitin machines have been proposed, both by Chaitin himself. Concrete algorithmic information theory involves the study of particular universal Turing machines, about which one can state theorems with specific numerical bounds, rather than include terms like O(1). We present several new tiny Chaitin machines (those with a prefix-free domain) suitable for the study of concrete algorithmic information theory. One of the machines, which we call Keraia, is a binary encoding of lambda calculus based on a curried lambda operator. Source code is included in the appendices. We also give an algorithm for restricting the domain of blank-endmarker machines to a prefix-free domain over an alphabet that does not include the endmarker; this allows one to take many universal Turing machines and construct universal Chaitin machines from them. * Strictly speaking, even the equals operator is just syntactic sugar: lambda terms are anonymous functions. † The combinator I is included merely for convenience; it can, of course, be replaced by SKK.

In this article, we investigate limitations of importing methods based on algorithmic information theory from monoplex networks into multidimensional networks (such as multilayer networks) that have a large number of extra dimensions (i.e., aspects). In the worst-case scenario, it has been previously shown that node-aligned multidimensional networks with non-uniform multidimensional spaces can display exponentially larger algorithmic information (or lossless compressibility) distortions with respect to their isomorphic monoplex networks, so that these distortions grow at least linearly with the number of extra dimensions. In the present article, we demonstrate that node-unaligned multidimensional networks, either with uniform or non-uniform multidimensional spaces, can also display exponentially larger algorithmic information distortions with respect to their isomorphic monoplex networks. However, unlike the node-aligned non-uniform case studied in previous work, these distortions in the node-unaligned case grow at least exponentially with the number of extra dimensions. On the other hand, for node-aligned multidimensional networks with uniform multidimensional spaces, we demonstrate that any distortion can only grow up to a logarithmic order of the number of extra dimensions. Thus, these results establish that isomorphisms between finite multidimensional networks and finite monoplex networks do not preserve algorithmic information in general and highlight that the algorithmic information of the multidimensional space itself needs to be taken into account in multidimensional network complexity analysis.

We show in this article that uncomputability is also a relative property of subrecursive classes built on a recursive relative incompressible function, which acts as a higher-order "yardstick" of irreducible information for the respective sub-recursive class. We define the concept of a Turing submachine, and a recursive relative version for the Busy Beaver function and for the halting probability (or Chaitin's constant) Ω; respectively the Busy Beaver Plus (BBP) function and a time-bounded halting probability. Therefore, we prove that the computable BBP function defined on any Turing submachine is neither computable nor compressible by any program running on this submachine. In addition, we build a Turing submachine that can use lower approximations to its own time-bounded halting probability to calculate the values of its Busy Beaver Plus function, in the " same " manner that universal Turing machines use approximations to Ω to calculate Busy Beaver values. Thus, the algorithmic information carried by the BBP function is relatively incompressible (and uncomputable) at the same time that it still is occasionally reached by submachines. We point that this phenomenon enriches the research on the relativization and simulation of uncomputability and irreducible information.

Previous work has shown that perturbation analysis in software space can produce candidate computable generative models and uncover possible causal properties from the finite description of an object or system quantifying the algorithmic contribution of each of its elements relative to the whole.
One of the challenges for defining emergence is that one observer's prior knowledge may cause a phenomenon to present itself to such observer as emergent while for another as reducible.
By formalising the act of observing as mutual perturbations between dynamical systems,
we demonstrate that emergence of algorithmic information do depend on the observer's formal knowledge, while robust to other subjective factors, particularly: the choice of the programming language and the measurement method; errors or distortions during the information acquisition; and the informational cost of processing.
This is called observer-dependent emergence (ODE).
In addition, we demonstrate that the unbounded and fast increase of emergent algorithmic information implies asymptotically observer-independent emergence (AOIE).
Unlike ODE, AOIE is a type of emergence for which emergent phenomena will remain considered to be emergent for every formal theory that any observer might devise.
We demonstrate the existence of an evolutionary model that displays the diachronic variant of AOIE and a network model that displays the holistic variant of AOIE.
Our results show that, restricted to the context of finite discrete deterministic dynamical systems, computable systems, and irreducible information content measures, AOIE is the strongest form of emergence that formal theories can attain.

In this paper, we analyze the problem of prediction in physics from the computational viewpoint. We show that physical paradigms like Laplace determinism, statistical determinism, etc., can be naturally explained by this computational analysis. In our explanations, we use the notions of the Algorithmic Information Theory such as Kolmogorov complexity and algorithmic randomness, as well as the novel, more physics-oriented variants of these notions.

As one of the main subjects of investigation in data science, network science has been demonstrated a wide range of applications to real-world networks analysis and modeling. For example, the pervasive presence of structural or topological characteristics, such as the small-world phenomenon, small-diameter, scale-free properties, or fat-tailed degree distribution were one of the underlying pillars fostering the study of complex networks. Relating these phenomena with other emergent properties in complex systems became a subject of central importance. By introducing new implications on the interface between data science and complex systems science with the purpose of tackling some of these issues, in this article we present a model for a network game played by complex networks in which nodes are computable systems. In particular, we present and discuss how some network topological properties and simple local communication rules are able to generate a phase transition with respect to the emergence of incompressible data.