Analysis of Continuous
Outcome Data - Means
1
2. Hypothesis testing
2.1. Single population mean
Ho: = o Versus s HA : ≠ o
,
Variance Unknown: the test is based on ‘t’ where:
x o
t
s/ n
Variance known: the test is based on ‘Z’ where:
x o
z
/ n
2.2. Two Population means
• Paired or Independent
• Two samples are said to be Paired when each
data point of the first sample is matched and is
related to a unique data point of the second
sample.
• Two samples are said to be independent if the
data points in one sample are unrelated to the
data points in the second sample
The paired t- test
• Hypothesis: Ho: = 0 versus HA: 0
• The test statistic (t) is given by:
t=
where and are the sample mean & standard
deviation of the observed difference from the paired
data
Independent population with equal
variances
• Hypothesis: Ho: 1 = 2 versus HA: 12
• The test statistic is given by:
X1 X 2
t
1 1
Sp
n1 n2
where, the weighted average of and could simply used
as the estimate of
• The degree of freedom will be the sum of the degree of
freedom of the two samples
Comparison of several means
6
Introduction to the One-way Analysis of variance
Frequently, the means of more than two distributions
are compared. The t-test methodology generalizes
nicely in this case to a procedure referred to as the
One-way Analysis of Variance (ANOVA)
7
The hypothesis to be tested for k means (k > 2)
H 0 : 1 2 3 ...
HA: At least one population mean is different from
another population mean
If we reject the null hypothesis, we conclude that
at least one population mean does not equal to
another population mean. Other methods are
needed to determine which population means are
different.
8
Examples
One-way analysis of variance
Social class levels (I to V) and blood
pressure
Type of sickle cell anaemia (3 Types) and
haemoglobin levels
Examine lipid levels between different ethnic
groups
9
Why not just use lots of t-tests?
• Practically time consuming
• Theoretically unsound - lots of tests would be
carried out
• Lead to false significant finding as 1 in 20 of
cases will be significant by chance
10
ANOVA answers the question
Are the differences in the means of the
different treatment (sample) groups likely to
simply reflect random variation associated
with sampling process
Or are these differences in the group means
too large to be consistent with the null
hypothesis?
11
Research Question
Does type of treatment (3 types) affect levels
of red blood cell folate?
Null Hypothesis
• There is no difference in mean levels of
red blood cell folate between the three
groups
12
Data - Red blood cell folate levels (µg/l)
Treatment Group
Treatment
I II III group Mean
243 206 241
251 210 258 I 316.6
275 226 270
II 256.4
291 249 293
347 255 328
III 278.0
354 273
380 285 Overall 283.2
392 295
309 13
Summary values by treatment groups
Red cell folate levels
Treatment No of Mean SD Individual values
group patients (xi) (si) x
(ni)
1 8 316.6 58.7 243,251, 275, 291, 347, 354,
380, 392
2 9 256.4 37.1 206, 210, 226, 249, 255, 273,
285, 295, 309
3 5 278.0 33.8 241, 258, 270, 293, 328
Overall 22 283.2 51.3
14
Analysis of Variance Table
Source of Sum of Degrees of Mean F Ratio
Variation Squares Freedom Square
Between 15515.8 2 7757.9 3.71
Groups P = 0.044
Residual or 39716.1 19 2090.3
Within
Total 55231.9 21
15
Reminder of variance around a mean
16
Notations
X ij Observation of the i th factor for the jth individual
ni
X i . X ij Total of the observations for the i th factor level
j 1
k ni
X
i 1 j 1
ij
X .. Overall mean for all responses
nT
ni
X
j 1
ij
Xi. Sample mean of the i th factor level
ni
17
Total Sum of squares
Total sum of squares (SST) – the numerator of the
equation for the variance
18
Partitioning the SST
k ni k ni
SST ( X ij X ..)2 ( X ij X i . X i . X ..)2
i 1 j 1 i 1 j 1
k ni
SST (( X ij X i .) ( X i . X ..)) 2
i 1 j 1
k k ni
SST ni ( X i . X ..)2 ( X ij X i .)2
i 1 i 1 j 1
k
SSB ni ( X i . X ..)2
i 1
k ni
SSW ( X ij X i .)2
i 1 j 1
19
Total Sum of Squares can be divided into:
• The variability within each treatment
(group)
• The variability between groups
20
Total Sum of Squares Partitioning
1. Sum of squares due to difference
between groups
2. Sum of squares due to differences
between observations within a group
(also known as residual sum of squares)
SST = SSB + SSW
21
Total sum of squares
SST = ΣΣ(x – x) 2 = 55,231.9
22
Within group sum of squares
SSW = 39,716.1
1. Find the difference between each individual
value and the mean of its group – (243-316.6),
(251-316.6), … (293-278), (328-278)
2. Square this difference
3. Sum across all groups
23
Table 1
Treatment No of Red cell folate levels
group patients
(ni) Mean SD Individual values
(xi) (si)
x
1 8 316.6 56.7 243,251, 275, 291, 347,
354, 380, 392
2 9 256.4 37.1 206, 210, 226, 249,
255, 273, 285, 295, 309
3 5 278 33.8 241, 258, 270, 293, 328
Overall 22 283.2 51.3
24
Between group sum of squares or
residual sum of squares
SSB = 15,515.8
1. Find the difference between each group mean
and the overall mean for the entire sample –
(316.6-283.2), (256.4-283.2), (278-283.2)
2. Square these differences
3. Multiply each squared difference by its group’s
sample size
4. Sum across all groups
25
Red Cell folates summary by treatment
Red cell folate levels
Treatment No of Mean SD Individual values
group patients (xi) (si) x
(ni)
1 8 316.6 56.7 243,251, 275, 291, 347, 354, 380,
392
2 9 256.4 37.1 206, 210, 226, 249, 255, 273, 285,
295, 309
3 5 278 33.8 241, 258, 270, 293, 328
Overall 22 283.2 51.3
26
Calculations
Source of Sum of Mean
df F-Ratio
Variation Squares Square
Between 15515.8 k-1 SS/df Between group MS
Within group MS
Groups
Residual 39716.1 n-k SS/df
(within)
Total 55231.9 n-1
27
Degrees of Freedom
What is it?
• Statisticians use the terms "degrees of
freedom" to describe the number of values
in the final calculation of a statistic that are
free to vary
28
Degrees of Freedom
• While there will be n such squared deviations
only (n - 1) of them are, in fact, free to assume
any value whatsoever
• Final squared deviation from the mean must
include the one value of X such that the sum of all
the Xs divided by n will equal mean of the sample
• All of the other (n - 1) squared deviations from the
mean can, theoretically, have any values
whatsoever.
• Therefore, the degree of freedom for SST = n-1
29
Degrees of Freedom
dftotal = (n1 + n2 + … nk) -1 = n-1 = 22-1 = 21
(ni = number of people in the ith group)
dfbetween = k-1 = 3-1 = 2 (k = No. of groups)
dfwithin = (n1-1) + (n2-1)+ …(nk-1) = n-k = 22-3 = 19
dftotal = dfbetween + dfwithin = 19 +2 = 21
30
Mean square
MSbetween = 15515.8/2 = 7757.9
MSwithin = 39716.1/19 = 2090.3
31
F-test or Variance ratio test
• If the null hypothesis is true that we would
expect the F-test from the null hypothesis to be
1
32
P-value
• Determined by using both degrees of
freedom (between and within group) and a
look up table for the F distribution
• More simply, we allow SPSS to calculate it
33
Analysis of Variance Table
Source of Sum of Degrees of Mean F Ratio
Variation Squares Freedom Square
Between 15515.8 2 7757.9 3.71
Groups P = 0.044
Residual 39716.1 19 2090.3
(within)
Total 55231.9 21
34
In SPSS
• Analyze – Compare means – One way Anova
35
Analysis of Variance table in SPSS
Descriptives
rcfol
95% Confidence Interval for
Mean
N Mean Std. Deviation Std. Error Lower Bound Upper Bound Minimum Maximum
1.00 8 316.6250 58.71709 20.75963 267.5363 365.7137 243.00 392.00
2.00 9 256.4444 37.12180 12.37393 227.9101 284.9788 206.00 309.00
3.00 5 278.0000 33.75648 15.09636 236.0858 319.9142 241.00 328.00
Total 22 283.2273 51.28439 10.93387 260.4890 305.9655 206.00 392.00
ANOVA
rcfol
Sum of
Squares df Mean Square F Sig.
Between Groups 15515.766 2 7757.883 3.711 .044
Within Groups 39716.097 19 2090.321
Total 55231.864 21
Conclusion: At least one of the groups means
differs from the others by more than we would
expect by sampling variation (p=0.044)
36
Multiple comparison tests
• Once an Analysis of Variance (ANOVA) test has
been completed, the researcher may still need
to understand subgroup differences among the
different experimental and control groups. The
subgroup differences are called “pairwise”
differences.
• ANOVA does not provide tests of pairwise
differences. When the researcher needs to test
pairwise differences, follow-up tests called post
hoc tests are required.
37
Multiple comparison tests
• This is because using t-tests to examine pairwise
differences is likely to overestimate the size of
the individual t-tests.
• As a result, performing multiple t-tests will lead
the researcher to a higher probability of making a
Type I error.
• That is, the researcher is much more likely to
report significant differences between some of
the pairs that have no real difference
38
Comparison of means
If F-test is significant, p<0.05, then you can
either:
• Carry out t-tests on pre-planned
comparisons of interest
• Combine treatments into similar groups,
then compare groups using t-test
• Use a multiple comparisons procedure
39
Bonferroni test
• Uses t tests to perform pairwise comparisons
between group means, but controls overall error
rate by setting the error rate for each test to the
experiment wise error rate () divided by the total
number of tests.
• Hence, the observed significance level () is
adjusted for the fact that multiple comparisons
are being made.
• Where k is the number of comparison groups
• The Bonferroni correction adjusts probability (p)
values because of the increased risk of a Type I
error when making multiple statistical tests.
40
The pairwise comparison using t-test is
(Unadjusted):
• Groups 1 & 2 p=0.029
• Groups 2 & 3 p=0.305
• Groups 1 & 3 p=0.161
41
Bonferroni t test
k=3 comparison α* = 0.05/ 3 = 0.016, p<0.016
Hypothesis α* = α / 3 = 0.05/3
H0 : μ1 = μ2 0.0167
H0 : μ1 = μ3 0.0167
H0 : μ2 = μ3 0.0167
Overall 0.05
42
Bonferroni t-test
In SPSS, it adjusts t statistics and therefore the p-
value and the result is:
• Groups 1 & 2 p= 0.042,
• Groups 2 & 3 p= 1.0
• Groups 1 & 3 p= 0.464
The p-value must be less than the adjusted α to
reject H0
The method should only be used with a small
number of comparisons
43
Multiple comparison testing
• Bonferroni test
• Tukey Test
• Dunnett’s test for multiple comparisons against a
single control group
• Student-Newman-Keuls Test for all pairwise
comparison
• Scheffee
• Duncan
• Debate in the statistics world on which is best
method
44
Multi-comparison in SPSS
45
Example of Bonferroni t-statistic for
multiple comparisons in SPSS
Multiple Comparisons
Dependent Variable: rcfol
Bonferroni
Mean
Difference 95% Confidence Interval
(I) treatg (J) treatg (I-J) Std. Error Sig. Lower Bound Upper Bound
1.00 2.00 60.18056* 22.21594 .042 1.8614 118.4998
3.00 38.62500 26.06443 .464 -29.7969 107.0469
2.00 1.00 -60.18056* 22.21594 .042 -118.4998 -1.8614
3.00 -21.55556 25.50141 1.000 -88.4995 45.3884
3.00 1.00 -38.62500 26.06443 .464 -107.0469 29.7969
2.00 21.55556 25.50141 1.000 -45.3884 88.4995
*. The mean difference is significant at the .05 level.
46
Assumptions
• Data is normally distributed for each group:
You may use the Normal Q–Q Plot to
assess the normality of each group, check
each Q-Q Plot lie close to their respective
diagonal lines
• The variance of the data (spread) is similar
in each of the groups: “Homogeneity-of-
Variance” (Levene’s test checks the
assumption of equal variances)
47
Normal Q-Q Plots of Folate levels by
Treatment
48
Checking model assumptions in linear
regression
Normality check: Plot a histogram of the
residuals (should be normally distributed)
5
1
Std. Dev = .95
Mean = 0.00
0 N = 22.00
-1.50 -1.00 -.50 0.00 .50 1.00 1.50
Standardized Residual for RCF
49
Histogram of standardised residuals
Similar spread of data – Homoscedasticity
Plot residuals against exposure variable (should be random
scatter of points - a sky at night plot)
50.00
Residual for rcfol
0.00
-50.00
1.00 1.50 2.00 2.50 3.00
50
treatg
Test for homogeneity of variance
• Under the options in ANOVA model, tick test for
homogeneity of variance
• If significant, the spread of the data is significantly
different between the groups
• Therefore non-parametric tests should be used
Test of Homogeneity of Variances
rcfol
Levene
Statistic df1 df2 Sig.
3.823 2 19 .040
51
Summary of one-way ANOVA
• Examine means (SD) in each group
• Check assumptions
• Conducted a one-way ANOVA
• If significant, then precede to pre-planned
multi-comparison testing
• Present results from multi-comparison
testing
52
Exercise
• Study conducted among men of age 18-25 years
in community. Perform appropriate analysis to
assess the effect SES on mean BMI
Lower SES Middle SES Higher SES
18 22 25
17 25 26
18 24 24
19 26 28
19 24 29
21
53
