23
2.2. Multivariate Probability Distributions
(I) Bivariate Probability Distribution
Joint Probability Distribution (Density) Function:
Let X 1 and X 2 be discrete random variables. The joint probability
distribution function is
f x1 , x 2 P X 1 x1 , X 2 x2 ,
where x1 and x 2 are possible values of X 1 and X 2 , respectively. f
satisfies the following conditions:
1. f x1 , x 2 0 for all x1 , x 2 ;
2. f x , x 1 .
x2 x1
1 2
Let X 1 and X 2 be continuous random variables. The joint probability
density function satisfies the following conditions:
1. f x1 , x 2 0 for x1 , x 2 ;
2. , x 2 dx1 d x 2 1 .
f x
1
Cumulative Distribution Function:
For any random variables X 1 and X 2 , the joint (cumulative) distribution
function is given by F x1 , x 2 P X 1 x1 , X 2 x2 .
As X 1 and X 2 are discrete,
P a X 1 b, c X 2 d f x ,x . 1 2
c x2 d a x1 b
As X 1 and X 2 are continuous,
d b
P a X 1 b, c X 2 d f x1 , x2 dx1dx2 .
c a
For any random variables X 1 and X 2 with joint cumulative distribution
function F x1 , x 2 ,
1. F , F , x 2 F x1 , 0;
2. F , 1 ;
3. if b a, d c , then
P a X 1 b, c X 2 d F b, d F b, c F a, d F a, c .
Example 1:
Form a collection of 3 white balls, 2 black balls, and 1 red ball, 2 balls is to be
randomly selected. Let X 1 denote the number of white balls and X 2 the
number of black balls.
(a) Find the joint probability distribution table of X 1 and X 2 .
23
24
(b) F 1,1 and F 2,0 .
[solution:]
(a)
3 2 1
i j 2i j
P X 1 i, X 2 j f i, j ,0 i, j 2; i j 1 or 2;
6
2
For example,
3 2 1
1 1 0 3 2 6 .
P X 1 1, X 2 1 f 1,1
6 15 15
2
The joint probability distribution table is
x1
x2 0 1 2 Total
0 0 3 3 6
15 15 15
1 2 6 0 8
15 15 15
2 1 0 0 1
15 15
Total 3 9 3 1
15 15 15
(b) F 1,1 P X 1 1, X 2 1 f 0,0 f 0,1 f 1,0 f 1,1 0 2 3 6 11 .
15 15
033 6
F 2,0 P X 1 2, X 2 0 f 0,0 f 1,0 f 2,0
15 15
Example 2:
Let f x1 , x 2 2 x1 ,0 x1 k ;0 x 2 1; and f x1 , x 2 0, otherwise.
(a) Find k .
(b) Find F 0.7,0.5 , F 2,0 and F 0.2,3 .
[solution:]
(a)
x
1 k 1
f x1 , x 2 dx1 dx 2
1
2 k
2 x1dx1dx2 dx 2 k dx 2 k 2 1 k 1 .
2
1 0 0
0 0 0
24
25
k 1.
(b)
0.5 0.7 0.50.7
F 0.7,0.5 P X 1 0.7, X 2 0.5 f x1 , x2 dx1dx2 2 x dx dx1 1 2
0 0
x
0.5 0.5
2 0.7
0
1 0 dx2 0.49dx
0
2 0.49 0.5 0.245
0 2 0 1
F 2,0 P X 1 2, X 2 0 f x1 , x2 dx1dx2 2 x1dx1dx2
0 0
0 0
1
x12 0 dx2 1dx2 1 0 0
0 0
3 0.2 1 0.2
F 0.2,3 P X 1 0.2, X 2 3 f x1 , x2 dx1dx2 2 x dx dx
1 1 2
0 0
x
1 1
2 0.2
1 0 dx2 0.04dx2 0.04
0 0
Marginal Probability Distribution (Density) Function:
Let X 1 and X 2 be discrete random variables. The marginal probability
distribution function for X 1 and X 2 are
f1 x1 P X 1 x1 f x1 , x 2 ,
x2
and
f 2 x 2 P X 2 x 2 f x1 , x 2 ,
x1
respectively.
Let X 1 and X 2 be continuous random variables. The marginal
probability density function for X 1 and X 2 are
f 1 x1 f x 1 , x 2 dx 2 ,
and
f 2 x2 f x 1 , x 2 dx1 ,
respectively.
Conditional Probability Distribution (Density) Function:
Let X 1 and X 2 be discrete random variables. The conditional probability
distribution function X 1 given X 2 is
P X 1 x1 , X 2 x 2 f x1 , x 2 ,
f1 x1 | x 2 P X 1 x1 | X 2 x 2
P X 2 x2 f 2 x2
25
26
provided that f 2 x 2 0 . Similarly, the conditional probability distribution
function X 2 given X 1 is
P X 1 x1 , X 2 x2 f x1 , x 2 ,
f 2 x 2 | x1 P X 2 x 2 | X 1 x1
P X 1 x1 f1 x1
provided that f 1 x1 0 .
Let X 1 and X 2 be continuous random variables. The conditional
probability density function X 1 given X 2 is
f x1 , x 2
, f 2 x2 0
f1 x1 | x 2 f 2 x 2
0, otherwise.
Similarly, the conditional probability density function X 2 given X 1 is
f x1 , x 2
, f1 x1 0
f 2 x 2 | x1 f 1 x1
0, otherwise.
Example 1 (continue):
(c) Find the marginal probability distribution function of X 1 .
(d) Given one of chosen balls being black, what is the distribution for the number
of balls being white?
[solution:]
(c) f1 (0) f 0,0 f 0,1 f 0,2 0 2 1 3 ,
15 15
3 6 0 9 ,
f 1 (1) f 1,0 f 1,1 f 1,2
15 15
300 3
f1 (2) f 2,0 f 2,1 f 2,2
15 15
(d) f 2 (1) f 0,1 f 1,1 f 2,1 2 6 0 8
15 15
2
f 0,1 15 1 ,
f1 0 | 1 P X 1 0 | X 2 1
f 2 1 8 4
15
26
27
6
f 1,1 15 3 ,
f1 1 | 1 P X 1 1 | X 2 1
f 2 1 8 4
15
f 2,1 0
f1 2 | 1 P X 1 2 | X 2 1 0
f 2 1 8
15
Example 2 (continue):
(c) Find the marginal probability density function for X 1 and X 2 .
(d) Find the conditional probability density function for X 1 given X 2 and the
conditional probability density function for X 2 given X 1
[solution:]
(c)
1
f1 x1 f x1 , x 2 dx 2 2 x1 dx 2 2 x1 ,0 x1 1; .
0
1
f 2 x2 f x , x dx
1
1 2 1 2 x1 dx1 x12 0
1,0 x 2 1;
0
(d) For 0 x1 1,0 x 2 1,
f x1 , x 2
f 1 x1 | x 2 2 x1
f 2 x2
and
f x1 , x 2 .
f 2 x2 | x1 1
f1 x1
Independence of Two Random Variables:
Let X 1 and X 2 be random variables. If X 1 and X 2 are independent if and
only if
f x1 , x 2 f1 x1 f 2 x 2 ,
for all pairs of x1 , x 2 .
Note: As f 2 x 2 0 and X 1 and X 2 are independent, then
f1 x1 | x 2 f1 x1 .
Further, as X 1 and X 2 are independent,
27
28
P X 1 x1 , X 2 x 2 f x1 , x 2 f1 x1 f 2 x 2 P X 1 x1 P X 2 x 2
P A B P A P B , A X 1 x1 , B X 2 x 2 .
Example 1 (continue):
(e) Are X 1 and X 2 independent?
[solutions:]
No since f 1,0 3 5 6 9 6 f1 1 f 2 0 .
15 25 25 15 15
Example 2 (continue):
(e) Are X 1 and X 2 independent?
[solutions:]
Yes since f x1 , x 2 2 x1 f1 x1 f 2 x 2 .
Example 3:
Let and Are
f x1 , x 2 6 x1 x 22 ,0 x1 k ;0 x 2 1; f x1 , x 2 0, otherwise. X1
and X 2 independent?
[solution:]
1
f 1 x1 f x1 , x 2 dx2 6 x1 x 22 dx 2 2 x1 x 23
1
0
0
2 x1 ,0 x1 1;
1
f 2 x2 f x1 , x 2 dx1 6 x1 x 22 dx1 3x12 x 22
1
0
0
3 x 22 ,0 x1 1.
Thus,
.
f x1 , x 2 6 x1 x 22 f1 x1 f 2 x 2
Therefore, X 1 and X 2 are independent.
(II) Multivariate Probability Distribution
Joint Probability Distribution (Density) Function:
Let X 1 , , X n be discrete random variables. The joint probability
distribution function is
f x1 , , x n P X 1 x1 , , X n x n ,
where x1 , , x n are possible values of X 1 , , X n , respectively. f
satisfies the following conditions:
1. f x1 , , x n 0 for all x1 , , x n ;
2. f x , , x 1 .
x2 xn
1 n
28
29
Let X 1 , , X n be continuous random variables. The joint probability
density function satisfies the following conditions:
1. f x1 , , x n 0 for x1 , , x n ; ;
2. , , x n dx1 d x n 1 .
f x
1
Independence of Multiple Random Variables:
Let X 1 , , X n be random variables. If X 1 , , X n are independent if and only if
f x1 , , x n f1 x1 f n x n ,
for all pairs of x1 , , x n , where f j is the marginal probability distribution (or
density) function of X j .
Likelihood Function:
Let x1 , , x n be sample observations taken on corresponding random variables
X 1 , , X n .whose joint probability distribution or density function depends on
the parameters 1 , , k . The likelihood of sample is
l 1 , , k f x1 , , x n | 1 , , k .
As X 1 , , X n are a random sample from a common probability distribution (or
density) with the parameters 1 , , k , i.e., the probability distribution or
density function equal to f x | 1 , , k and X 1 , , X n being i.i.d. (identically
independent distributed), the likelihood is
n
l 1 , , k f xi | 1 , , k f x1 | 1 , , k f xn | 1 , , k .
i 1
Example 4:
Let X 1 , , X n are a random sample from a normal distribution N , 2 . Then,
n
xi 2 xi 2
l , 2 2 2
n 2 2
e n i 1
2 2
2
e
i 1 2 2
Example 5:
Let X 1 , , X n are a random sample from a binomial distribution B m, p .
Then,
n
n
m xi n m xi n
nm xi
l p p 1 p p 1 p i 1
m xi i 1
i 1 xi i 1 xi
Example 6:
Let X 1 , , X n are a random sample from a Poisson distribution Poisson .
Then,
29
30
n
xi
n
e xi e n i 1
l n
xi !
i 1
x!
i 1
i
30
