Multivariate Probability
Distributions: Part III
Cyr Emile MLAN, Ph.D.
mlan@[Link]
Multivariate Probability Distributions: Part III
p. 1/1
Introduction
Text Reference: Introduction to Probability and Its
Applications, Chapter 6.
Reading Assignment: Sections 6.6, April 20
We discuss now two special multivariate models: the
multinomial distribution and the bivariate normal
distribution.
Multivariate Probability Distributions: Part III
p. 2/1
Multinomial Distributions
The multinomial distribution is an important generalization of the
binomial distribution for situations where each of n independent trials
can result in one of k distinct outcomes.
Multinomial Experiment
Definition 6.14:
A multinomial experiment possesses the following properties:
The experiment consists of a fixed number n of identical trials.
The outcome of each trial falls into one of k classes or cells.
The probability that the outcome of a single trial will fall in a
particular cell, say cell i, is pi , (i = 1, , k) and remains the
same from trial to trial. Note that p1 + p2 + + pk = 1.
The trials are independent.
The random variable of interest are X1 , X2 , , Xk where
Xi , i = 1, , k denotes the random variable representing the
Multivariate Probability Distributions: Part III
p. 3/1
number of trials falling into cell i. Note that X1 +X2 + +Xk = n.
Multinomial Distributions, cont.
Joint Probability Mass Function
The joint probability mass function for (X1 , X2 , , Xk )
is given by
k
Y
n!
pixi ,
P (X1 = x1 , X2 = x2 , . . . , Xk = xk ) =
x1 !x2 ! . . . xk !
i=1
where y1 + y2 + + yk = n and p1 + p2 + + pk = 1.
Notation
We use the following notation for the multinomial
distribution:
X1 , X2 , , Xk Mult n, (p1 , p2 , . . . , pk ) .
Multivariate Probability Distributions: Part III
p. 4/1
Marginal Distributions of Multinomial Distributions
Properties
(1) Suppose
X1 , X2 , , Xk
Mult n, (p1 , p2 , . . . , pk ) .
then each Xi follows a binomial distribution:
Xi Binomial(n, pi )
for i = 1, 2, . . . , k
Multivariate Probability Distributions: Part III
p. 5/1
Expectations and Covariances of Multinomial Distributions
Expectations and Covariances
Theorem 6.8:
If X1 , X2 , X3 , . . . , Xk have the multinomial distribution with parameters n and p1 , p2 , , pk , then
E[Xi ] = npi
Var(Xi ) = npi (1 pi )
Cov(Xi , Xj ) = npi pj , i 6= j.
Multivariate Probability Distributions: Part III
p. 6/1
Expected Values and Covariances of Multinomial Distributions
Example 6.15:
A sample of size n is selected from a large lot of items
in which a proportion p1 contain one defect and a
proportion p2 contain two or more defects. Some of the
items had no defects. The cost of repairing the defective
items is C = X1 + 3X2 where X1 denotes the number of
items with one defect and X2 denotes the number of
items with two or more defects. Find E(C) and Var(C).
Solution:
We have:
E(C) = E(X1 ) + 3E(X2 )
= n(p1 + 3p2 )
Var(C) = Var(X1 ) + 9Var(X2 ) + 6Cov(X1 , X2 )
= n [p1 (1 p1 ) + 9p2 (1 p2 ) 6p1 p2 ]
Multivariate Probability Distributions: Part III
p. 7/1
Bivariate Normal Distributions
Let X1 and X2 be two random variables with means
1 = E[X1 ] and 2 = E[X2 ], variances 12 = Var(X1 ) and
22 = Var(X2 ), and covariance
12 = Cov(X1 , X2 ) = 1 2 .
X1 and X2 forms a bivariate normal distribution if their
joint probability density function can be written as
(
"
1
(x1 1 )2
f (x1 , x2 ) =
exp
2(1 2 )
12
21 2 1 2
#)
(x1 1 ) (x2 2 ) (x2 2 )2
+
.
2
2
1
2
2
1
p
for < x1 < and < x2 <
Multivariate Probability Distributions: Part III
p. 8/1
Marginal Distributions of Bivariate Normal Distributions
Properties
(1) The marginal distributions of X1 and X2 are still
normal.
More specifically, we have
X1 N (1 , 12 ) and X2 N (2 , 22 ).
(2) If = 0 then X1 and X2 are independent Normal
random variables and vice-versa.
More specifically, we have
f (x1 , x2 ) = f1 (x1 ) f2 (x2 )
if and only if
= 0.
This is not true in general for other bivariate
distributions.
Multivariate Probability Distributions: Part III
p. 9/1
Conditional Distributions of Bivariate Normal Distribution
(3) The conditional distributions are also normal.
More specifically, the conditional distribution of X1
given X2 = x2 is
1
X1 | X2 = x2 N 1 + (x2 2 ), (1 2 )12 .
2
Similarly, the conditional distribution of X2 given
X1 = x1 is
2
X2 | X1 = x1 N 2 + (x1 1 ), (1 2 )22 .
1
Multivariate Probability Distributions: Part III
p. 10
