STA 532: Theory of Statistical Inference

The nature of stochastic dependence in the classic bivariate normal density framework is analyzed. In the case of this distribution, we stress the way the conditional density of one of the random variables depends on realizations of the other. Typically, in the bivariate normal case this dependence takes the form of a parameter (here the 'expected value') of one probability density depending continuously (here linearly) on realizations of the other random variable. Our point is that such a pattern does not need to be restricted to that classical case of the bivariate normal. We show that this paradigm can be generalized, and viewed in ways that allows us to extend it far beyond the bivariate normal distributions class.

Mathematics and Statistics, 2021

The present article derives the minimal number N of observations needed to approximate a Bayesian posterior distribution by a Gaussian. The derivation is based on an invariance requirement for the likelihood p(xj). This requirement is defined by a Lie group that leaves the p(xj) unchanged, when applied both to the observation(s) x and to the parameter  to be estimated. It leads, in turn, to a class of specific priors. In general, the criterion for the Gaussian approximation is found to depend on (i) the Fisher information related to the likelihood p(xj), and (ii) on the lowest non-vanishing order in the Taylor expansion of the Kullback-Leibler distance between p(xj) and p(xjML), where ML is the maximum-likelihood estimator of , given by the observations x. Two examples are presented, widespread in various statistical analyses. In the first one, a chi-squared distribution, both the observations x and the parameter  are defined all over the real axis. In the other one, the binomial distribution, the observation is a binary number, while the parameter is defined on a finite interval of the real axis. Analytic expressions for the required minimal N are given in both cases. The necessary N is an order of magnitude larger for the chi-squared model (continuous x) than for the binomial model (binary x). The difference is traced back to symmetry properties of the likelihood function p(xj). We see considerable practical interest in our results since the normal distribution is the basis of parametric methods of applied statistics widely used in diverse areas of research (education, medicine, physics, astronomy etc.). To have an analytical criterion whether the normal distribution is applicable or not, appears relevant for practitioners in these fields.

arXiv (Cornell University), 2019

The problem of characterizing a multivariate distribution of a random vector using examination of univariate combinations of vector components is an essential issue of multivariate analysis. The likelihood principle plays a prominent role in developing powerful statistical inference tools. In this context, we raise the question: can the univariate likelihood function based on a random vector be used to provide the uniqueness in reconstructing the vector distribution? In multivariate normal (MN) frameworks, this question links to a reverse of Cochran's theorem that concerns the distribution of quadratic forms in normal variables. We characterize the MN distribution through the univariate likelihood type projections. The proposed principle is employed to illustrate simple techniques for assessing multivariate normality via well-known tests that use univariate observations. The presented testing strategy can exhibit high and stable power characteristics in comparison to the well-known procedures in various scenarios when observed vectors are non-MN distributed, whereas their components are normally distributed random variables. In such cases, the classical multivariate normality tests may break down completely.

Journal of Multivariate Analysis, 1991

Couallier/Statistical Models and Methods for Reliability and Survival Analysis, 2013

This paper presents a new approach to conditional inference, based on the simulation of samples conditioned by a statistics of the data. Also an explicit expression for the approximation of the conditional likelihood of long runs of the sample given the observed statistics is provided. It is shown that when the conditioning statistics is sufficient for a given parameter, the approximating density is still invariant with respect to the parameter. A new Rao-Blackwellisation procedure is proposed and simulation shows that Lehmann Scheffé Theorem is valid for this approximation. Conditional inference for exponential families with nuisance parameter is also studied, leading to Monte carlo tests. Finally the estimation of the parameter of interest through conditional likelihood is considered. Comparison with the parametric bootstrap method is discussed.

Annals of the Institute of Statistical Mathematics, 2012

This paper introduces a new family of local density separations for assessing robustness of finite-dimensional Bayesian posterior inferences with respect to their priors. Unlike for their global equivalents, under these novel separations posterior robustness is recovered even when the functioning posterior converges to a defective distribution, irrespectively of whether the prior densities are grossly misspecified and of the form and the validity of the assumed data sampling distribution. For exponential family models, the local density separations are shown to form the basis of a weak topology closely linked to the Euclidean metric on the natural parameters. In general, the local separations are shown to measure relative roughness of the prior distribution with respect to its corresponding posterior and provide explicit bounds for the total variation distance between an approximating posterior density to a genuine posterior. We illustrate the application of these bounds for assessing robustness of the posterior inferences for a dynamic time series model of blood glucose concentration in diabetes mellitus patients with respect to alternative prior specifications.

Canadian Journal of Statistics, 1985

Some recent discussions of the logic involved in statistical inference have focussed on the given (i.e. the statistical model and the data) and the role of the common reduction principles (namely conditionality, likelihood, and sufficiency). The minimum statistical model, a class of probability measures on a measurable space, can yield many different density-function models with consequent arbitrariness for the definition of the likelihood function and with anomalies in the presence of the reduction principles; instances are cited. Restrictions are given for the minimum model; these lead to a canonical definition for density models and for likelihood functions, and to modified definitions for conditionality and sufficiency to avoid the anomalies. RESUME Ces temps derniers, les dkbats que soulevent sans cesse I'infkrence statistique et ses assises rationnelles ont eu davantage tendance a porter sur ce que I'on pourrait convenir d'appeler les ostensibles, c'est-a-dire les donnkes, le modele qui les sous-tend, ainsi que I'influence des principes reducteurs coutumiers, a savoir I'exhaustivitk, la conditionalisation et le principe de vraisemblance. Dans cet article, nous allons plus loin pour ne considkrer que ce qui constitue le strict minimum d'un modele statistique: un espace mesurable conjugut a une classe de mesures de probabilitk. Comme on peut choisir diffkrentes manieres d'exprimer ces mesures de probabilitk sous forme de densitCs, la fonction de vraisemblance souffre fatalement d'un certain arbitraire, ce qui provoque parfois des anomalies lorsque I'on invoque les principes rtducteurs citks plus haut. En plus de substantier cet Cnonct, nous suggkrons ici la maniere d'tviter cette embiiche en exigeant de tout modele statistique qu'il satisfasse certaines conditions additionnelles. Ces conditions conduisent a une dkfinition canonique des fonctions de vraisemblance et des modeles qui font intervenir des fonctions de densitk. Pour tviter toute anomalie, les dkfinitions des principes de conditionalisation et d'exhaustivitk doivent Cgalement Ctre modifikes. Les Cltments structurels requis pour le traitement d'autres modeles seront considtrCs ailleurs.

Statistics & Probability Letters, 1994

If X is a k-dimensional random vector, we denote by X(i,j) the vector X with coordinates i and j deleted. If for each i, j the conditional distribution of Xi, Xj given X(i,j) = x(i,j) is classical bivariate normal for each then it is shown that X has a classical k-variate normal distribution.

Canadian Journal of Statistics, 1994

Definitions are given for orthogonal parameters in the context of Bayesian inference and likelihood inference. The exact orthogonalizing transformations are derived for both cases, and the connection between the two settings is made precise. These parametrizations simplify the interpretation of likelihood functions and posterior distributions. Further, they make numerical maximization and integration procedures easier to apply. Several applications are studied. RESUME Nous prksentons des dkfinitions pour des paramitres orthogonaux dans le contexte de I'infkrence de Bayes et de I'infkrence de vraisemblance. Les transformations d'orthogonalisation exactes sont obtenues dans les deux cas et le lien entre les deux approches est prCcise. Ces paramktrisations simplifient I'interprktation des fonctions de vraisemblance et des distributions a posteriori. En outre, elles rendent I'application des prockdures de maximisation numerique et d'intkgration plus facile. Quelques applications sont ktudikes.