Sequentially interacting Markov chain Monte Carlo methods

1997

We study the slice sampler, a method of constructing a reversible Markov chain with a specified invariant distribution. Given an independence Metropolis-Hastings algorithm it is always possible to construct a slice sampler that dominates it in the Peskun sense. This means that the resulting Markov chain produces estimates with a smaller asymptotic variance. Furthermore the slice sampler has a smaller second-largest eigenvalue than the corresponding independence MetropolisHastings algorithm. This ensures faster convergence to the distribution of interest. A sufficient condition for uniform ergodicity of the slice sampler is given and an upper bound for the rate of convergence to stationarity is provided. Keywords: Auxiliary variables, Slice sampler, Peskun ordering, Metropolis-Hastings algorithm, Uniform ergodicity. 1 Introduction The slice sampler is a method of constructing a reversible Markov transition kernel with a given invariant distribution. Auxiliary variables ar...

Data Mining and Knowledge Discovery, 2003

Markov chain Monte Carlo (MCMC) techniques revolutionized statistical practice in the 1990s by providing an essential toolkit for making the rigor and flexibility of Bayesian analysis computationally practical. At the same time the increasing prevalence of massive datasets and the expansion of the field of data mining has created the need for statistically sound methods that scale to these large problems. Except for the most trivial examples, current MCMC methods require a complete scan of the dataset for each iteration eliminating their candidacy as feasible data mining techniques. In this article we present a method for making Bayesian analysis of massive datasets computationally feasible. The algorithm simulates from a posterior distribution that conditions on a smaller, more manageable portion of the dataset. The remainder of the dataset may be incorporated by reweighting the initial draws using importance sampling. Computation of the importance weights requires a single scan of the remaining observations. While importance sampling increases efficiency in data access, it comes at the expense of estimation efficiency. A simple modification, based on the “rejuvenation” step used in particle filters for dynamic systems models, sidesteps the loss of efficiency with only a slight increase in the number of data accesses. To show proof-of-concept, we demonstrate the method on two examples. The first is a mixture of transition models that has been used to model web traffic and robotics. For this example we show that estimation efficiency is not affected while offering a 99% reduction in data accesses. The second example applies the method to Bayesian logistic regression and yields a 98% reduction in data accesses.

Computational Statistics & Data Analysis, 2012

A novel class of interacting Markov chain Monte Carlo (MCMC) algorithms, hereby referred to as the Parallel Hierarchical Sampler (PHS), is developed and its mixing properties are assessed. PHS algorithms are modular MCMC samplers designed to produce reliable estimates for multi-modal and heavy-tailed posterior distributions. As such, PHS aims at benefitting statisticians whom, working on a wide spectrum of applications, are more focused on defining and refining models than constructing sophisticated sampling strategies. Convergence of a vanilla PHS algorithm is proved for the case of Metropolis-Hastings within-chain updates. The accuracy of this PHS kernel is compared with that of optimized single-chain and multiple-chain MCMC algorithms for multimodal mixtures of multivariate Gaussian densities and for 'banana-shaped' heavytailed multivariate distributions. These examples show that PHS can yield a dramatic improvement in the precision of MCMC estimators over standard samplers. PHS is then applied to two realistically complex Bayesian model uncertainty scenarios. First, PHS is used to select a low number of meaningful predictors for a Gaussian linear regression model in the presence of high collinearity. Second, the posterior probability of survival trees approximated by PHS indicates that the number and size of liver metastases at the time of diagnosis are predictive of substantial differences in the survival distributions of colorectal cancer patients.

A critical issue for users of Markov Chain Monte Carlo (MCMC) methods in applications is how to determine when it is safe to stop sampling and use the samples to estimate characteristics of the distribution of interest. Research into methods of computing theoretical convergence bounds holds promise for the future but currently has yielded relatively little that is of practical use in applied work. Consequently, most MCMC users address the convergence problem by applying diagnostic tools to the output produced by running their samplers. After giving a brief overview of the area, we provide an expository review of thirteen convergence diagnostics, describing the theoretical basis and practical implementation of each. We then compare their performance in two simple models and conclude that all the methods can fail to detect the sorts of convergence failure they were designed to identify. We thus recommend a combination of strategies aimed at evaluating and accelerating MCMC sampler convergence, including applying diagnostic procedures to a small number of parallel chains, monitoring autocorrelations and cross-correlations, and modifying parameterizations or sampling algorithms appropriately. We emphasize, however, that it is not possible to say with certainty that a finite sample from an MCMC algorithm is representative of an underlying stationary distribution.

Sadhana, 2006

Markov Chain Monte Carlo (MCMC) is a popular method used to generate samples from arbitrary distributions, which may be specified indirectly. In this article, we give an introduction to this method along with some examples.

Cornell University - arXiv, 2019

Markov chain Monte Carlo (MCMC) methods generate samples that are asymptotically distributed from a target distribution of interest as the number of iterations goes to infinity. Various theoretical results provide upper bounds on the distance between the target and marginal distribution after a fixed number of iterations. These upper bounds are on a case by case basis and typically involve intractable quantities, which limits their use for practitioners. We introduce L-lag couplings to generate computable, non-asymptotic upper bound estimates for the total variation or the Wasserstein distance of general Markov chains. We apply L-lag couplings to the tasks of (i) determining MCMC burn-in, (ii) comparing different MCMC algorithms with the same target, and (iii) comparing exact and approximate MCMC. Lastly, we (iv) assess the bias of sequential Monte Carlo and self-normalized importance samplers.

1998

Markov chain Monte Carlo (MCMC) methods, including the Gibbs sampler and the Metropolis-Hastings algorithm, are very commonly used in Bayesian statistics for sampling from complicated, high-dimensional posterior distributions. A continuing source of uncertainty is how long such a sampler must be run in order to converge approximately to its target stationary distribution. Rosenthal (1995b) presents a method to compute rigorous theoretical upper bounds on the number of iterations required to achieve a specified degree of convergence in total variation distance by verifying drift and minorization conditions. We propose the use of auxiliary simulations to estimate the numerical values needed in Rosenthal's theorem. Our simulation method makes it possible to compute quantitative convergence bounds for models for which the requisite analytical computations would be prohibitively difficult or impossible. On the other hand, although our method appears to perform well in our example problems, it can not provide the guarantees offered by analytical proof.

2000

Markov chain Monte Carlo (MCMC) methods have become popular as a basis for drawing inference from complex statistical models. Two common di culties with MCMC algorithms are slow convergence and long run-times, which are often closely related. Algorithm convergence can often be aided by careful tuning of the chain's transition kernel. In order to preserve the algorithm's stationary distribution, however, care must be taken when updating a chain's transition kernel based on that same chain's history. In this paper we introduce a technique that allows the transition kernel to be updated at user speci ed intervals, while preserving the chain's stationary distribution. This technique may be bene cial in aiding both the rate of convergence (by allowing adaptation of the transition kernel) and the speed of computing. The approach is particularly helpful when calculation of the full conditional (for a Gibbs algorithm) or of the candidate distribution (for a Metropolis-Hastings algorithm) is computationally expensive.

2010

Let P(E) be the space of probability measures on a measurable space (E, E). In this paper we introduce a class of non-linear Markov Chain Monte Carlo (MCMC) methods for simulating from a probability measure π ∈ P(E). Non-linear Markov kernels (e.g. Del ; Del ) K : P(E) × E → P(E) can be constructed to admit π as an invariant distribution and have superior mixing properties to ordinary (linear) MCMC kernels. However, such non-linear kernels cannot be simulated exactly, so, in the spirit of particle approximations of Feynman-Kac formulae (Del Moral 2004), we construct approximations of the non-linear kernels via Self-Interacting Markov Chains (Del Moral & Miclo 2004) (SIMC). We present several non-linear kernels and demonstrate that, under some conditions, the associated self-interacting approximations exhibit a strong law of large numbers; our proof technique is via the Poisson equation and Foster-Lyapunov conditions. We investigate the performance of our approximations with some simulations, combining the methodology with population-based Markov chain Monte Carlo (e.g. Jasra et al. ). We also provide a comparison of our methods with sequential Monte Carlo samplers when applied to a continuous-time stochastic volatility model.

In many problems, complex non-Gaussian and/or nonlinear models are required to accurately describe a physical system of interest. In such cases, Monte Carlo algorithms are remarkably flexible and extremely powerful approaches to solve such inference problems. However, in the presence of a high-dimensional and/or multimodal posterior distribution, it is widely documented that standard Monte-Carlo techniques could lead to poor performance. In this paper, the study is focused on a Sequential Monte-Carlo (SMC) sampler framework, a more robust and efficient Monte Carlo algorithm. Although this approach presents many advantages over traditional Monte-Carlo methods, the potential of this emergent technique is however largely underexploited in signal processing. In this work, we aim at proposing some novel strategies that will improve the efficiency and facilitate practical implementation of the SMC sampler specifically for signal processing applications. Firstly, we propose an automatic an...

Particle Markov Chain Monte Carlo methods are used to carry out inference in non-linear and non-Gaussian state space models, where the posterior density of the states is approximated using particles. Current approaches have usually carried out Bayesian inference using a particle Metropolis-Hastings algorithm or a particle Gibbs sampler. In this paper, we give a general approach to constructing sampling schemes that converge to the target distributions given in Andrieu et al. [2010] and Olsson and Ryden [2011]. We describe our methods as a particle Metropolis within Gibbs sampler (PMwG). The advantage of our general approach is that the sampling scheme can be tailored to obtain good results for different applications. We investigate the properties of the general sampling scheme, including conditions for uniform convergence to the posterior. We illustrate our methods with examples of state space models where one group of parameters can be generated in a straightforward manner in a particle Gibbs step by conditioning on the states, but where it is cumbersome and inefficient to generate such parameters when the states are integrated out. Conversely, it may be necessary to generate a second group of parameters without conditioning on the states because of the high dependence between such parameters and the states. Our examples include state space models with diffuse initial conditions, where we introduce two methods to deal with the initial conditions.

ASCE-ASME J. Risk and Uncert. in Engrg. Sys., Part B: Mech. Engrg., 2017

The transitional Markov chain Monte Carlo (TMCMC) is one of the efficient algorithms for performing Markov chain Monte Carlo (MCMC) in the context of Bayesian uncertainty quantification in parallel computing architectures. However, the features that are associated with its efficient sampling are also responsible for its introducing of bias in the sampling. We demonstrate that the Markov chains of each subsample in TMCMC may result in uneven chain lengths that distort the intermediate target distributions and introduce bias accumulation in each stage of the TMCMC algorithm. We remedy this drawback of TMCMC by proposing uniform chain lengths, with or without burn-in, so that the algorithm emphasizes sequential importance sampling (SIS) over MCMC. The proposed Bayesian annealed sequential importance sampling (BASIS) removes the bias of the original TMCMC and at the same time increases its parallel efficiency. We demonstrate the advantages and drawbacks of BASIS in modeling of bridge dy...

2009

This paper introduces the Parallel Hierarchical Sampler (PHS), a class of Markov chain Monte Carlo algorithms using several interacting chains having the same target distribution but different mixing properties. Unlike any single-chain MCMC algorithm, upon reaching stationarity one of the PHS chains, which we call the “mother” chain, attains exact Monte Carlo sampling of the target distribution of interest. We

1998

The formal convergence diagnosis of the Markov Chain Monte Carlo (MCMC) is made using univariate and multivariate criteria. In 1998, a multivariate extension of the univariate criterion of multiple sequences was proposed. However, due to some problems of that multivariate criterion, an alternative form of calculation was proposed in addition to the two new alternatives for multivariate convergence criteria. In this study, two models were used, one related to time series with two interventions and ARMA (2, 2) error and another related to a trivariate normal distribution, considering three different cases for the covariance matrix. In both the cases, the Gibbs sampler and the proposed criteria to monitor the convergence were used. Results revealed the proposed criteria to be adequate, besides being easy to implement.

Journal of the American Statistical Association, 1998

I propose a convergence diagnostic for Markov Chain Monte Carlo (MCMC) algorithms that is based on couplings of a Markov chain with an auxiliary chain that is periodically restarted from a xed parameter value. The diagnostic provides a mechanism for estimating the speci c constants governing the rate of convergence of geometrically and uniformly ergodic chains, and provides an informal lower bound on the \e ective" sample size of a MCMC chain of xed length. It also provides a simple procedure for obtaining what is, with high probability, an independent sample from the stationary distribution.