(Non-) Equivalence of Universal Priors
Ian D Wood
Peter Sunehag
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
"Abstract. Ray Solomonoff invented the notion of universal induction featuring an aptly termed “universal” prior probability function over all possible computable environments [9]. The essential property of this prior was its ability to dominate all other such priors. Later, Levin introduced another construction — a mixture of all possible priors or “universal mixture” [12]. These priors are well known to be equivalent up to mul- tiplicative constants. Here, we seek to clarify further the relationships between these three characterisations of a universal prior (Solomonoff’s, universal mixtures, and universally dominant priors). We see that the the constructions of Solomonoff and Levin define an identical class of priors, while the class of universally dominant priors is strictly larger. We provide some characterisation of the discrepancy. "
Related papers
In principle determination of generic priors
Probability theory as extended logic is completed such that essentially any probability may be determined. This is done by considering propositional logic (as opposed to predicate logic) as syntactically sufficient and imposing a symmetry from propositional logic. It is shown how the notions of 'possibility' and 'property' may be sufficiently represented in propositional logic such that 1) the principle of indifference drops out and becomes essentially combinatoric in nature and 2) one may appropriately represent assumptions where one assumes there is a space of possibilities but does not assume the size of the space.
Non-zero probabilities for universal generalizations
Synthese, 2013
This article discusses the classical problem of zero probability of universal generalizations in Rudolf Carnap's inductive logic. A correction rule for updating the inductive method on the basis of evidence will be presented. It will be shown that this rule has the effect that infinite streams of uniform evidence assume a non-zero limit probability. Since Carnap's inductive logic is based on finite domains of individuals, the probability of the corresponding universal quantification changes accordingly. This implies that universal generalizations can receive positive prior and posterior probabilities, even for (countably) infinite domains.
A new criterion for prior probabilities
2010
Howson and Urbach (1996) wrote a carefully structured book supporting the Bayesian view of scientific reasoning, which includes an unfavorable judgment about the so-called objective Bayesian inference. In this paper, the theses of the book are investigated from Carnap's analytical viewpoint in the light of a new formulation of the Principle of Indifference. In particular, the paper contests the thesis according to which no theory can adequately represent 'ignorance' between alternatives. Beginning from the new formulation of the principle, a criterion for the choice of an objective prior is suggested in the paper together with an illustration for the case of Binomial sampling. In particular, it will be shown that the new prior provides better frequentist properties than the Jeffreys interval.
A Note on Commensurate Universals in the Posterior Analytics
Phronesis 24, 1979
A consideration of the role of commensurately universal (i.e., convertible) propositions in Aristotle's theory of explanation in the Post. An.
On the non-arbitrary assignment of equi-probable priors
2007
How to form priors that do not seem artificial or arbitrary is a central question in Bayesian statistics. The case of forming a prior on the truth of a proposition for which there is no evidence, and the definte evidence that the event can happen in a finite set of ways, is detailed. The truth of a propostion of this kind is frequently assigned a prior of 0.5 via arguments of ignorance, randomness, the Principle of Indiffernce, the Principal Principal, or by other methods. These are all shown to be flawed. The statistical syllogism introduced by Williams in 1947 is shown to fix the problems that the other arguments have. An example in the context of model selection is given.
Objective and subjective foundations for multiple priors
Journal of Economic Theory
Foundations for priors can be grouped in two broad categories: objective, deriving probabilities from observations of similar instances; and subjective, deriving probabilities from the internal consistency of choices. Partial observations of similar instances and the Savage-de Finetti extensions of subjective priors yield objective and subjective sets of priors suitable for modeling choice under ambiguity. These sets are best suited to such modeling when the distribution of the observables, or the prior to be extended, is nonatomic. In this case, the sets can be used to model choices between elements of the closed convex hull of the faces in the set of distributions over outcomes, equivalently, all sets bracketed by the upper and lower probabilities induced by correspondences.
A more robust definition of multiple priors
Carlo Alberto Notebook, 2010
This paper provides a multiple-priors representation of ambiguous beliefs à la Ghirardato, for any preference that is (i) monotonic, (ii) Bernoullian, i.e. admits an affine utility representation when restricted to constant acts, and (iii) suitably continuous. Monotonicity is the main substantive assumption:
On logical and mereological renderings of the Bayes theorem (short paper)
2019
1 The Bayes theorem published posthumously as the work of Rev. Thomas Bayes (1701/2-1761) in ‘Essay Towards Solving a Problem in the Doctrine of Chances’ (1764) rediscovered by Lagrange, provides a foundation for some areas of Artificial Intelligence like Bayesian Reasoning, Bayesian Filtering etc. It had been reformulated in logical terms by Jan Lukasiewicz (1913). Recently, an abstract version couched in mereological terms was formulated and a strengthening of it appeared derived from the Stone representation theorem for complete Boolean algebras. It is our aim to comprehensively present those approaches with emphasis on the abstract setting of mass assignments on mereological universes endiwed with rough inclusions induced by masses of things. 1 The Bayes theorem original and the rendering by Lukasiewicz Given a probability distribution on a space of events Ω (cf.[5]) one defines the conditional probability P (E|H) of the event E modulo the event H as P (E|H) = P (E∩H) P (H) . Fr...
Direct Inference and Probabilistic Accounts of Induction
Journal for General Philosophy of Science
Schurz (2019, ch. 4) argues that probabilistic accounts of induction fail. In particular, he criticises probabilistic accounts of induction that appeal to direct inference principles, including subjective Bayesian approaches (e.g., Howson 2000) and objective Bayesian approaches (see, e.g., Williamson 2017). In this paper, I argue that Schurz’ preferred direct inference principle, namely Reichenbach’s Principle of the Narrowest Reference Class, faces formidable problems in a standard probabilistic setting. Furthermore, the main alternative direct inference principle, Lewis’ Principal Principle, is also hard to reconcile with standard probabilism. So, I argue, standard probabilistic approaches cannot appeal to direct inference to explicate the logic of induction. However, I go on to defend a non-standard objective Bayesian account of induction: I argue that this approach can both accommodate direct inference and provide a viable account of the logic of induction. I then defend this ac...
Understanding Common Priors under Incomplete Information
The Common Prior Assumption (CPA) plays an important role in game theory and the economics of information. It is the basic assumption behind decision-theoretic justifications of equilibrium reasoning in games (Aumann, 1987, Aumann and Brandenburger, 1995) and no-trade results with asymmetric information (Milgrom and Stokey, 1982). Recently several authors (Dekel and Gul, 1997, Gul, 1996, Lipman, 1995) have questioned whether the CPA is meaningful in situations of incomplete information, where there is no ex ante stage and where the primitives of the model are the individuals' beliefs about the external world (their first-order beliefs), their beliefs about the other individuals' beliefs (second-order beliefs), etc., i.e. their hierarchies of beliefs. In this context, the CPA is a mathematical property whose conceptual content is not clear. The main results of this paper (Theorems 1 and 2) provide a characterization of Harsanyi consistency in terms of properties of the belief hierarchies that are entirely unrelated to the idea of an ex ante stage.
Related topics
Related papers
The co∼event-based Bayes' theorem (arXiv.org version)
The logic of uncertainty is not the logic of experience and as well as it is not the logic of chance. It is the logic of experience and chance. Experience and chance are two inseparable poles. These are two dual reections of one essence, which is called co∼event. The theory of experience and chance is the theory of co∼events. To study the co∼events, it is not enough to study the experience and to study the chance. For this, it is necessary to study the experience and chance as a single entire, a co∼event. In other words, it is necessary to study their interaction within a co∼event. The new co∼event axiomatics and the theory of co∼events following from it were created precisely for these purposes. In this work, I am going to demonstrate the effectiveness of the new theory of co∼events in a studying the logic of uncertainty. I will do this by the example of a co∼event splitting of the logic of the Bayesian scheme, which has a long history of erce debates between Bayesionists and frequentists. I hope the logic of the theory of experience and chance will make its modest contribution to the application of these old dual debaters., theory of experience and chance, co∼event dualism, co∼event axiomatics, logic of uncertainty, logic of experience and chance, logic of cause and consequence, logic of the past and the future, Bayesian scheme.
How common are common priors?
Games and Economic Behavior, 2012
To answer the question in the title we vary agents' beliefs against the background of a fixed knowledge space, that is, a state space with a partition for each agent. Beliefs are the posterior probabilities of agents, which we call type profiles. We then ask what is the topological size of the set of consistent type profiles, those that are derived from a common prior (or a common improper prior in the case of an infinite state space). The answer depends on what we term the tightness of the partition profile. A partition profile is tight if in some state it is common knowledge that any increase of any single agent's knowledge results in an increase in common knowledge. We show that for partition profiles that are tight the set of consistent type profiles is topologically large, while for partition profiles that are not tight this set is topologically small.
A characterization of a neutral to the right prior via an extension of Johnson’s sufficientness postulate
The Annals of Statistics, 1999
On the axiomatic approach to the maximum entropy principle of inference
Pramana, 1986
Recent axiomatic derivations of the maximum entropy principle from consistency conditions are critically examined. We show that proper application of cons'mtency conditions alone allows a wider class of functionals, essentially of the form S dx p(x) [p(x)/o(x)]', for some real number s, to be used for inductive inference and the commonly used formS dx p (x) In [p (x)/o (x)] is only a particular case. The role of the prior density 0 (x) is clarified. It is possible to regard it as a geometric factor, describing the coordinate system used and it does not represent information of the same kind as obtained by measurements on the system in the form of expectation values.
On the foundations of Bayesianism
AIP Conference Proceedings, 2001
We discuss precise assumptions entailing Bayesianism in the line of investigations started by Cox, and relate them to a recent critique by Halpern. We show that every finite model which cannot be rescaled to probability violates a natural and simple refinability principle. A new condition, separability, was found sufficient and necessary for rescalability of infinite models. We finally characterize the acceptable ways to handle uncertainty in infinite models based on Cox's assumptions. Certain closure properties must be assumed before all the axioms of ordered fields are satisfied. Once this is done, a proper plausibility model can be embedded in an ordered field containing the reals, namely either standard probability (field of reals) for a real valued plausibility model, or extended probability (field of reals and infinitesimals) for an ordered plausibility model. The end result is that if our assumptions are accepted, all reasonable uncertainty management schemes must be based on sets of extended probability distributions and Bayes conditioning.
On the Probabilistic Convention T
Review of Symbolic Logic, 2008
We introduce an epistemic theory of truth according to which the same rational degree of belief is assigned to Tr('alpha') and alpha. It is shown that if epistemic probability measures are only demanded to be finitely additive (but not necessarily sigma-additive), then such a theory is consistent even for object languages that contain their own truth predicate. As the proof of this result indicates, the theory can also be interpreted as deriving from a quantitative version of the Revision Theory of Truth.
On the invariance of noninformative priors
The Annals of Statistics, 1996
The generalized universal law of generalization
Journal of Mathematical Psychology, 2003
It has been argued by Shepard that there is a robust psychological law that relates the distance between a pair of items in psychological space and the probability that they will be confused with each other. Specifically, the probability of confusion is a negative exponential function of the distance between the pair of items. In experimental contexts, distance is typically defined in terms of a multidimensional Euclidean space-but this assumption seems unlikely to hold for complex stimuli. We show that, nonetheless, the Universal Law of Generalization can be derived in the more complex setting of arbitrary stimuli, using a much more universal measure of distance. This universal distance is defined as the length of the shortest program that transforms the representations of the two items of interest into one another: the algorithmic information distance. It is universal in the sense that it minorizes every computable distance: it is the smallest computable distance. We show that the universal law of generalization holds with probability going to one-provided the confusion probabilities are computable. We also give a mathematically more appealing form of the universal law.
Description and the Problem of Priors
Erkenntnis, 2014
Belief-revision models of knowledge describe how to update one's degrees of belief associated with hypotheses as one considers new evidence, but they typically do not say how probabilities become associated with meaningful hypotheses in the first place. Here we consider a variety of Skyrms-Lewis signaling game [Lewis (1969)] [Skyrms (2010)] where simple descriptive language and predictive practice and associated basic expectations coevolve. Rather than assigning prior probabilities to hypotheses in a fixed language then conditioning on new evidence, the agents begin with no meaningful language or expectations then evolve to have expectations conditional on their descriptions as they evolve to have meaningful descriptions for the purpose of successful prediction. The model, then, provides a simple but concrete example of how the process of evolving a descriptive language suitable for inquiry might also provide agents with effective priors.
On the relative belief transform
International Journal of Approximate Reasoning, 2012
In this paper we discuss the semantics and properties of the relative belief transform, a probability transformation of belief functions closely related to the classical plausibility transform. We discuss its rationale in both the probability-bound and Shafer's interpretations of belief functions. Even though the resulting probability (as it is the case for the plausibility transform) is not consistent with the original belief function, an interesting rationale in terms of optimal strategies in a non-cooperative game can be given in the probability-bound interpretation to both relative belief and plausibility of singletons. On the other hand, we prove that relative belief commutes with Dempster's orthogonal sum, meets a number of properties which are the duals of those met by the relative plausibility of singletons, and commutes with convex closure in a similar way to Dempster's rule. This supports the argument that relative plausibility and belief transform are indeed naturally associated with the D-S framework, and highlights a classification of probability transformations in two families, according to the operator they relate to. Finally, we point out that relative belief is only a member of a class of \relative mass" mappings, which can be interpreted as low-cost proxies for both plausibility and pignistic transforms.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.