' $
ST3241 Categorical Data Analysis I
Generalized Linear Models
Some More Discussions
& %
1
' $
Deviance
• A saturated model has a separate parameter for each
observation giving a perfect fit.
• Let θ̃ denote the estimate of θ for the saturated model,
corresponding to estimated means µ̃i = yi for all i.
• Let θ̂ denote the MLE of θ for the model under consideration.
• The deviance of the fitted model is defined as
D(y; µ̂) = −2[L(µ̂; y) − L(y; y)]
N
X N
X
=2 [yi θ̃i − b(θ̃i )]/a(φ) − 2 [yi θ̂i − b(θ̂i )]/a(φ)
i=1 i=1
& %
2
' $
Deviance
• Usually, a(φ) has the form a(φ) = φ/wi , and this statistic
equals
N
X
2 wi [yi (θ̃i − θ̂i ) − b(θ̃i ) + b(θ̂i )]/φ
i=1
• This is called scaled deviance.
• The greater the scaled deviance, the poorer the fit.
• For some GLMs, the scaled deviance has an approximate
chi-squared distribution.
& %
3
' $
Deviance For Poisson Model
• For Poisson GLMs,
θ̂i = log µ̂i and b(θ̂i ) = exp(θ̂i ) = µ̂i
• Similarly, for saturated model
θ̃i = log yi and b(θ̃i ) = yi
• Also, a(φ) = 1 and the deviance is equal to
N
X
D(y; µ̂) = 2 [yi log(yi /µ̂i ) − yi + µ̂i ]
i=1
• When a model with log-link contains an intercept term, the
deviance simplifies to
N
X
D(y; µ̂) = 2 yi log(yi /µ̂i )
i=1
& %
4
' $
Deviance For Binomial Model
• Consider binomial GLMs with sample proportions {yi } based
on {ni } trials. Then
θ̂i = log[π̂i /(1 − π̂i )] and b(θ̂i ) = log[1 + exp(θ̂i )] = − log(1 − π̂i )
• Similarly, for the saturated model,
θ̃i = log[yi /(1 − yi )] and b(θ̃i ) = − log(1 − yi )
• Also, a(φ) = 1/ni , so φ = 1 and wi = ni . The deviance equals
P
N
yi π̂i
2 ni {yi (log 1−y i
− log 1−π̂i ) + log(1 − yi ) − log(1 − π̂i )}
i=1
P
N
ni y i P
N
ni −ni yi
=2 ni yi log ni π̂i +2 (ni − ni yi ) log ni −ni π̂i
i=1 i=1
• At setting i, ni yi is the number of successes and (ni − ni yi ) is
the number of failures, i = 1, · · · , N. Thus the deviance is
P
D(y; µ̂) = 2 observed × log(observed/f itted)
& %
5
' $
Deviance Residuals
• Define
di = 2wi [yi (θ̃i − θ̂i ) − b(θ̃i ) + b(θ̂i )]
• The deviance residual for observation i is
p
di × sign(yi − µ̂i )
& %
6
' $
Some SAS Codes
data glm;
input snoring disease total;
datalines;
0 24 1379
2 35 638
4 21 213
5 30 254
;
proc genmod; model disease/total = snoring / dist=bin
link=identity;
proc genmod; model disease/total = snoring / dist=bin
link=logit;
proc genmod; model disease/total = snoring / dist=bin
link=probit;
run;
& %
7
' $
Some R Codes
snoring<-c(0,2,4,5)
disease<-c(24,35,21,30)
total<-c(1379,638,213,254)
glm(cbind(disease,total-disease) snoring,
family=binomial(link="logit"))
glm(cbind(disease,total-disease) snoring,
family=binomial(link=probit"))
Reference: McCullagh, P. and Nelder, J.A.
(1989).Generalized Linear Models. 2nd ed.
London: Chapman and Hall.
& %
8
