Lecture 12

Joint Probability Distributions Two discrete R.Vs.

In the discrete case, f(x, y) = P(X = x and Y = y); that is, the values f(x, y) give the probability that
outcomes x and y occur at the same time.

Definition: The function f(x, y) is a joint probability distribution or joint probability mass
function of the discrete random variables X and Y if
1. f(x, y) ≥ 0 for all (x, y),
2. ∑∑ f ( x, y ) = 1,
x y

3. P(X = x, Y = y) = f(x, y).

For any region A in the xy plane, P[(X, Y) ∈ A] = ∑∑ f ( x, y ) .
A

Definition: The marginal distributions of X alone and of Y alone are

=g ( x) ∑=
f ( x, y ) and h( y ) ∑ f ( x, y )
y x

for the discrete case.

Statistical Independence
Definition: Let X and Y be two random variables, discrete or continuous, with joint probability
distribution f(x, y) and marginal distributions g(x) and h(y), respectively.
The random variables X and Y are said to be statistically independent if and only if
f(x, y) = g(x)h(y)
for all (x, y) within their range.
Definition: Let X 1 ,X 2 , . . . , X n be n random variables, discrete or continuous, with joint
probability distribution f(x 1 , x 2 , . . . , x n ) and marginal distribution f 1 (x 1 ), f 2 (x 2 ), . . . ,
f n (x n ), respectively. The random variables X 1 ,X 2 , . . . , X n are said to be mutually
statistically independent if and only if
f(x 1 , x 2 , . . . , x n ) = f 1 (x 1 )f 2 (x 2 ) · · · f n (x n )
for all (x 1 , x 2 , . . . , x n ) within their range.

PHM111s - Probability and Statistics

Example: The joint probability density function of two discrete random variables X and Y is
given by:
 c( x + y ), x=0,1,2 y=0,1,2,3,
f(x, y) = 
 0, otherwise.
Find
(a) c.
(b) P( X ≥ 1, Y ≤ 2) .
(c) g(x) and h(y)
(d) Check independence of the two random variables X and Y.

x Row Totals
f(x, y)
0 1 2
0 0 c 2c 3c (y = 0)
1 c 2c 3c 6c (y = 1)
y
2 2c 3c 4c 9c (y = 2)
3 3c 4c 5c 12c (y = 3)
6c 10c 14c ∑∑ f ( x, y ) = 30c
Column Totals
(x= 0) (x= 1) (x= 2) x y

2 3
1
Solution: (a) ∑∑
x y
f ( x, y ) =1 ⇒ ∑∑ c( x + y ) =1 ⇒ c =
=
x 0=
y 0 30
15
(b) P( X ≥ 1, Y ≤ 2) =
30
3
1
(c) g ( x=
) ∑ f ( x, y=) ∑ c( x + y=)
y y=0 15
(2 x + 3), =x 0,1,2
2
1
) ∑ f ( x, y=
h ( y= ) ∑ c ( x + y=
) ( y + 1), y= 0,1,2,3
x x=0 10
(d) f(0,0) ≠ g(0) h(0)

6 3
0 ⇒ X and Y are not independent.
30 30
Also, independence of the two random variables X and Y can be checked as follows:

f(x,y) ≠ g(x) h(y)

1 1 1
( x + y) (2 x + 3) ( y + 1) ⇒ X and Y are not independent.
30 15 10

PHM111s - Probability and Statistics

Definition: Let X and Y be random variables with joint probability distribution f(x, y). The mean,
or expected value, of the random variable g(X, Y) is

μ g(X,Y) = E[g(X, Y)] = ∑∑ g ( x, y ) f ( x, y )

x y
if X and Y are discrete.

Note that if g(X, Y) = X , we have

E(X) = ∑∑ xf ( x, y ) = ∑ xg ( x) (discrete case), where g(x) is the marginal distribution of X.

x y x

Similarly, we define

E(Y) = ∑∑ yf ( x, y ) = ∑ yh( y )
x y x
(discrete case), where h(y) is the marginal distribution Y.

Definition: Let X and Y be random variables with joint probability distribution f(x, y). The
covariance of X and Y is

σ XY = E[( X − µ X )(Y − µY )] = ∑∑ ( x − µ
x y
X
)( y − µY ) f ( x, y )

if X and Y are discrete.

Theorem: The covariance of two random variables X and Y with means µ X and µY , respectively,
is given by
Cov(X, Y) = σ XY = E ( XY ) − µ X µY = E ( XY ) − E ( X ) E (Y ).

• Cov(X, Y) = Cov(Y, X)
• Cov(X, c) = 0
• Cov(X, X) = Var (X)
• Cov(aX, bY) = ab Cov(X, Y)
• Cov(X ± a, Y ± b) = Cov(X, Y)

Var(aX+bY+c) = σ aX2 + bY + c = a 2σ X2 + b 2σ Y2 + 2abσ XY .

• σ= 2
aX + c
a=2
σ X2 a 2σ 2
• σ X2= +c
σ= 2
X
σ2
• =σ aX2 a= 2
σ X2 a 2σ 2
If X and Y are independent random variables, then
σ=2
aX ± bY
a 2σ X2 + b 2σ Y2 .
Theorem: Let X and Y be two independent random variables. Then E(XY) = E(X)E(Y).

Corollary: Let X and Y be two independent random variables. Then σ XY = 0.

PHM111s - Probability and Statistics
 Definition: Let X and Y be random variables with covariance σ XY and standard deviations σ X
and σ Y , respectively. The correlation coefficient of X and Y is

σ XY
ρ XY = . −1 ≤ ρ XY ≤ 1
σ Xσ Y

=σX E( X 2 ) − E 2 ( X ) σ XY E ( XY ) − E ( X ) E (Y )
= =σy E (Y 2 ) − E 2 (Y )

• ρ ( X , Y ) = ρ (Y , X )
• ρ(X , X ) = 1
• ρ (aX , bY ) = ρ ( X , Y )
• ρ ( X ± a, Y ± b) = ρ ( X ,Y )

Example 12.1: Two ballpoint pens are selected at random from a box that contains 3 blue pens, 2
red pens, and 3 green pens. If X is the number of blue pens selected and Y is the
number of red pens selected, find
(a) the joint probability function f(x, y),
(b) P[(X, Y) ∈ A], where A is the region {(x, y)|x + y ≤ 1}.
(c) g(x) and h(y) then check independence of the two random variables X and Y.
(d) the correlation coefficient between X and Y.
Solution: The possible pairs of values (x, y) are (0, 0), (0, 1), (1, 0), (1, 1), (0, 2), and (2, 0).

(a) Now, f(0,1), for example, represents Row

x
the probability that a red and a green f(x, y) Totals
pens are selected: 0 1 2
2 3 3 2 6 3 3 9 3 15
f(0, 1) = = × + × = = . 0
8 7 8 7 28 14 28 28 28 28
3 3 3
Similar calculations yield the y 1 0
14 14 7
probabilities for the other cases. 1 1
2 0 0
(b) The probability that (X, Y) fall in the 28 28
region A is Column 5 15 3 ∑∑ f ( x, y ) = 1
Totals 14 28 28 x y

P[(X, Y) ∈ A] = P(X + Y ≤ 1) = f(0, 0) + f(0, 1) + f(1, 0)

3 3 9 9
= + + = .
28 14 28 14
PHM111s - Probability and Statistics
 (c) For the random variable X, we see that
3 3 1 5
g(0) = f(0, 0) + f(0, 1) + f(0, 2) = + + =,
28 14 28 14
9 3 15
g(1) = f(1, 0) + f(1, 1) + f(1, 2) = + +0= ,
28 14 28
and
3 3
g(2) = f(2, 0) + f(2, 1) + f(2, 2) = +0+0= ,
28 28
which are just the column totals of the table. In a similar manner we could show that the
values of h(y) are given by the row totals. In tabular form, these marginal distributions
may be written as follows:

x 0 1 2 y 0 1 2
5 15 3 15 3 1
g(x) h(y)
14 28 28 28 7 28
From the table, we find the three probabilities f(0, 1), g(0), and h(1) to be
3
f(0, 1) = ,
14
2
3 3 1 5
g (0) = ∑ f (0, y ) = + + = ,
y=0 28 14 28 14
2
3 3 3
h(1) = ∑ f ( x,1) = + +0= .
x=0 14 14 7
Clearly, f(0, 1) ≠ g(0)h(1),

and therefore, X and Y are not statistically independent.

2
5 15 3 3
(d) E ( X ) == µ X ∑ xg ( x) = (0)( ) + (1)( ) + (2)( ) = ,
x=0 14 28 28 4
and
2
15 3 1 1
E (Y ) = µY = ∑
y=0
yh( y ) =
(0)( ) + (1)( ) + (2)( ) =.
28 7 28 2
 5  15   3  27
E ( X 2 ) = (02 )   + (12 )   + (22 )   =
 14   28   28  28
and
 5   3  1  4
E (Y 2 ) = (02 )   + (12 )   + (22 )   = ,
 28  
7  
28 7

PHM111s - Probability and Statistics

 Row
x
f(x, y) Totals
0 1 2
3 9 3 15
0
28 28 28 28
3 3 3
y 1 0
14 14 7
1 1
2 0 0
28 28
Column 5 15 3 ∑∑ f ( x, y ) = 1
Totals 14 28 28 x y

2 2
27  3  45 4 1 9
⇒ σ = −   = and σ Y2 =−   =.
2

28  4  112 7  2  28
X

2 2
E(XY) = ∑∑ xy f ( x, y )
=
x 0=
y 0

= (0)(0)f(0,0) + (0)(1)f(0, 1) + (0)(2)f(0,2) + (1)(0)f(1, 0) + (1)(1)f(1,1) + (1)(2)f(1,2)

+ (2)(0)f(2, 0) + (2)(1)f(2, 1) + (2)(2)f(2, 2)
3
= f(1, 1) = .
14
Therefore,

3 3 1 9
σ XY =
E ( XY ) − µ X µY =− ( )( ) =
− .
14 4 2 56
Therefore, the correlation coefficient between X and Y is
9
σ XY − 1
ρ XY = = 56 = − .
σ Xσ Y 45 9 5
( )( )
112 28

=
Example 12.14: If X and Y are random variables with variances σ X2 2=
and σ Y2 4 and
covariance σ XY = −2 , find the variance of the random variable Z = 3X − 4Y + 8.

Solution:
=σ Z2 σ=2
3 X − 4Y + 8
σ 32X − 4Y
=9σ X2 + 16σ Y2 − 24σ XY
= 9(2) + 16(4) − 24(−2)= 130.

PHM111s - Probability and Statistics

Example 12.15: Let X and Y denote the amounts of two different types of impurities in a batch
of a certain chemical product. Suppose that X and Y are independent random
=
variables with variances σ X2 2= and σ Y2 3 . Find the variance of the random
variable Z = 3X − 2Y + 5.

Solution:
=σ Z2 σ=
2
3 X − 2Y + 5
σ 32X − 2Y
= 9σ X2 + 4σ Y2
= 9(2) + 4(3) = 30.

PHM111s - Probability and Statistics