Lectured by

STT553: Categorical Data Analysis (CDA)

Md. Kaderi Kibria, STT-HSTU

Lecture # 4 Delta Method

Objectives of this lecture:

After reading this unit, you should be able to
• basic concepts of Two-way contingency table
• understand about the normal approximation
• understand about the delta method

Delta Method
In statistics, the delta method is a method of deriving the asymptotic distribution of a random
variable. It is applicable when the random variable being considered can be defined as a
differentiable function of a random variable which is asymptotically Gaussian.
When it is very difficult or impossible to find the mean and variance of a function f(x) of a random
variable x then we can obtain approximate mean and variance of f(x) with the help of Taylors series.
This method is called statistical differential or delta method.

Delta Method for a Function of a Random Variable

Let Tn denote a statistic, the subscript expressing it dependence on the sample size n. For large
sample, suppose Tn has approximately a normal distribution with mean θ and standard error
σ / √n . More precisely, as n tends to infinity, the cdf of √ n(T n−θ ) converge to a N (0 , σ 2)
cdf. i.e √ n(T n−θ )→N (0 , σ 2 )
For a function g, we now derive the limiting distribution of g (T n) . Suppose that g is at least
twice differential at θ .
Let
t=θ +t−θ
g (t )=g ( θ +t−θ )

Expanding g(t) by taylors series we get

2
g (t )=g ( θ )+(t −θ ) ǵ ( θ )+(t−θ ) ´ǵ ( θ )/2
2
=g ( θ )+(t −θ ) ǵ ( θ )+O (|t −θ |)

Replace t by Tn we have
2
g (T n)=g ( θ )+( T n−θ ) ǵ ( θ )+O (|T n−θ |)
2
g (T n)−g ( θ )=(T n −θ ) ǵ ( θ )+O(|T n−θ |)
√ n[ g (T n)−g (θ )]=√ n(T n−θ ) ǵ ( θ )+ √ n O(|T n −θ2|)
√ n[ g (T n)−g (θ )]=√ n(T n−θ ) ǵ ( θ ) [Since √ n 0(|T n−θ 2|)is asympotically negligible]

Categorical Data Analysis | Lecture 3 | Two-way Contingency table

 Lectured by
STT553: Categorical Data Analysis (CDA)
Md. Kaderi Kibria, STT-HSTU

So, √ n[ g (T n)−g (θ )] has the same limiting distribution as √ n(T n−θ ) ǵ ( θ ) that is,
g (T n)−g ( θ ) behaves like the constant multiple ǵ ( θ ) of (T n−θ ) . Now, (T n −θ ) is
approximately normal with variance σ 2 /n . Thus, g (T n)−g ( θ ) is approximately normal with
variance σ 2 [ ǵ ( θ )]2 /n . More precisely,

√ n[ g (T n)−g (θ )]→ N [0 , σ 2 ( ǵ ( θ )2)]

Since ǵ ( θ ) and σ 2=σ 2 ( θ ) usually depend on the unknown parameter θ , the asymptotic
variance is unknown. Confidence intervals and tests substitute T n for θ and use the result that

√ n [ g (T n)−g (θ )] /|ǵ (T n )|σ (T n) is asymptotically standard normal. For instance,

g (T n)±1.96|ǵ (T n)|σ (T n )/ √ n

is a large-sample Wald 95% confidence interval for g ( θ ).

Delta Method Applied to Sample Logit

The delta method for a function of the ML estimator T n=π^ = y /n of the binomial parameter
π , for y success in n trails.
So,
E ( y )=n π and Var ( y )=n π (1− π ) then
π (1−π )
E ( π^ )=π and Var ( π^ )=
n
Also, π^ has a large sample normal distribution by the central limit theorem.
The log odds function of π^
g ( π^ )=log[ π^ /(1− π^ )]

is called the sample logit.

By using the delta method, the asymptotic distribution of g ( π^ ) is

1
√ n[ g ( π^ )−g ( π )]→N 0 , [ n π (1−π ) ]
The asymptotic normality of π^ propagates to asymptotic normality of log [ π^ /(1− π^ ) ] .

Delta Method for log Odds Ratio

Standard errors for the log odds ratio from multi-parameter version of the delta method. Suppose
that {ni; i=1,2, . . . , c} have a multinomial (n , {πi }) distribution. The sample proportion
π^ =n i /n has mean and variance

E( π^ i)=πi and Var ( π^ i )=π i( 1−π i)/n

Categorical Data Analysis | Lecture 3 | Two-way Contingency table

 Lectured by
STT553: Categorical Data Analysis (CDA)
Md. Kaderi Kibria, STT-HSTU

We also previously show that for i≠ j , π^ i and π^ j have covariance

cov ( π^ i , π^ j )=− πi π j /n

The sample proportions ( π^ 1 , π^2 , . .. , π c−1

^ ) have a large sample multivariate normal distribution.
Let g ( π ) denote a differentiable function of {πi } with sample value g ( π^ ) for a multinomial
sample. Let
δ g ( π)
ϕ i= ; i=1,2 , . .. , c
δ πi
Then as n→ ∞ , the distribution of √ n[ g ( π^ )−g ( π )]/ σ converge to standard normal where
2
σ2 =∑ π ϕ 2i −( ∑ π ϕi )

The asymptotic variance depends on {πi } and the partial derivatives of the measure with respect
to {πi }. We now apply the delta method to the log odds ratio,

n11 n 22 π 11 π 22
Let g ( π )=log θ =log ( )=log ( π π )=log π11 + log π 22−log π12 −log π 21
n12 n21 12 21

Since,
1
ϕ 11 =δ log θ / δ π 11= π
11

1 1 1
Similarly, ϕ 12=− π ; ϕ21=− π ; and ϕ 22= π
12 21 22

1
then ∑ ∑ πij ϕij=0 and σ 2=∑ ∑ πij ϕij2 =∑ ∑ ( π ). The asymptotic standard error of
ij

log θ^ for a multinomial sample {nij } is

1
σ (log θ^ )=( ∑ ∑ 1/ πij ) 2

By using the delta method, the asymptotic distribution of g ( π^ ) is

2
√ n[ g ( π^ )−g ( π )]→N [ 0 , ( ∑ ∑ 1/ πij ) ]
Problem 1:

Solution: From the delta method, we know that

√ n[ g (T n)−g (θ )]→ N [0 , σ 2 ( ǵ ( θ )2)] where

Categorical Data Analysis | Lecture 3 | Two-way Contingency table

 Lectured by
STT553: Categorical Data Analysis (CDA)
Md. Kaderi Kibria, STT-HSTU

Given that
g ( π )= ν / δ where ν and δ are the function of π .
Now,
ni
ϕ i= ∂∂π g ( π )= ∂∂π ( ν ) δ2
=
i i δ

2
2
[ 2
Then, σ = ∑ πi ηi −(∑ πi ηi ) / δ ] 4.

Problem 2:

Solution:
(a) From the definition of marginal distribution.
π 1 + =π11 + π12=θ 2 + θ (1−θ )=θ
π +1 =π11 + π21=θ 2 + θ (1−θ )=θ
That means π1 + =π+1 , so the marginal distributions are identical.
Again, we know that if πij =πi + ×π+ j , then marginals are independent. Here,
π11 = π1 + . π +1=θ . θ= θ2
(b) The likelihood function becomes,
nij n11 n12 n21 n22
L=∏ πij =π 11 ×π12 ×π 21 ×π 22
Then the log likelihood function becomes,
log L=n11 log π11 + n12 log π12+ n21 log π21 +n 22 log π 22
= n11 log θ 2+(n 12+ n21) log [ θ ( 1−θ )]+ n22 log (1−θ )2
Now,
∂ log L 2 n 11 2
∂θ = θ +(n12 + n21)/ θ −( n12 +n 21)/(1− θ )+ n22 log (1−θ ) =0
2 n11 + n12+ n21 n + n +n +n n +n π +π
θ^ = = 11 12 11 21 = 1 + +1 = 1 + +1
2(n11 + n12+ n 21+ n22) 2n 2n 2
n n
where π 1 + = 1 + and π+ 1= +1
n n

Categorical Data Analysis | Lecture 3 | Two-way Contingency table

 Lectured by
STT553: Categorical Data Analysis (CDA)
Md. Kaderi Kibria, STT-HSTU

Reference Book:
i. Agresti A. (2019), An Introduction to Categorical Data Analysis, 3rd edition, A John Wiley & Sons
Inc., Publication.
ii.
<><><><><><><><><> End <><><><><><><><><>

Categorical Data Analysis | Lecture 3 | Two-way Contingency table