📌

Two-sample t-tests

📌
Lesson Structure
Independent Samples Interview Questions
Two-Sample T-test with Equal Variances
Calculate the mean difference and the
Welch’s T-test with Unequal Variances
standard error of mean difference for a two-
Dependent Samples
sample t-test
Advantages
Disadvantages Implement two-sample t-tests in Python/R.

Independent Samples

💡 X1 and X2 are random samples from two independent populations.

Each population is (approximately) normally distributed.

The samples within each population are independent of each other.

Independent/unpaired samples

There is no relationship between the subjects in each sample.

Subjects in the first group cannot also be in the second group

No subject in either group can influence subjects in the other group

No group can influence the other group

Hypotheses
H0 : μ1 = μ2 ,
H1 : μ1 
= μ2 .
Steps to conduct an independent t-test

Two-sample t-tests 1
 Calculate difference among sample means

Sum the variances of the samples and calculate the equal or unequal variances
Two scenarios:

Equal variances

Unequal variances

Calculate t-statistic and compare it with t-critical value

If |t-statistic| > t-critical value, reject H0 .

Two-Sample T-test with Equal Variances

When the two groups’ variances are believed to be equal, we use a t-test based on a pooled variance
estimate.

t-statistic

Xˉ1 − Xˉ2
T = ∼ tn1 +n2 −2,α/2
1 1
sp n1
+ n2

ˉ i is the sample mean of sample i, for i

Where X ∈ {1, 2}, and the pooled sample variance s2p is
defined by s2p = S S 1 +S S 2
with SSi = ∑nj=1
i
(Xj − X ˉ i )2 .
n1 +n2 −2

Confidence interval
A level (1 − α) ∗ 100% confidence interval for the difference between two means:

ˉ1 − X
ˉ 2 ± tn +n −2,α/2 ∗ sp 1 1
X 1 2
+
n1 n2

Welch’s T-test with Unequal Variances

If the two variances are not similar (one is more than twice of the other) but the other assumptions for the
2-sample t-test hold, then we can use Welch’s t-test, which employs an unpooled standard error.

t-statistic
s 21 s 22
Unpooled standard error: n1
+ n2

Xˉ1 − Xˉ2
T = ∼ tdf ,α/2
s 21 s 22
n1
+ n2

2 2
∑ ni=1 (X 1i − Xˉ1 ) ∑ ni=1 (X 2i − Xˉ2 )
where s1 = n1 −1 , s2 = n2 −1

and

⁍.

Two-sample t-tests 2
 E.g. Compare the mean heights of men in two cities.
We sample n1 = 19 and n2 = 23 men respectively. We simulate some data and run the test in
Python as follows. We assume that the two populations have different variances.

H0 : μ1 = μ2 ,
H1 : μ1 =
 μ2 .

We are in the rejection region so there is a statistically significant difference between the height of men
in the two cities.

Confidence interval

A level (1 − α) ∗ 100% confidence interval for the difference between two means:

ˉ1 − X
ˉ 2 ± tdf , α s21 s2
X + 2
2
n1 n2

2 2
∑ ni=1 (X 1i − Xˉ1 ) ∑ ni=1 (X 2i − Xˉ2 ) (s 21 /n1 +s 22 /n2 )2
where s1 = n1 −1
, s2 = n2 −1
and df = (s 21 /n1 )2 /(n1 −1)+(s 22 /n2 )2 /(n2 −1)
.

E.g. Confidence interval for the difference of heights of men.

Two-sample t-tests 3
 Dependent Samples
Dependent/paired samples: each data point in one sample is uniquely paired to a

data point in the second sample. Examples:

Change over time (longitudinal study), measure one variable at one point in time and measure the same
variable of the same sample at a later point in time.

Pre-test vs. post-test, to see whether there is a significant effect due to the treatment.

Advantages
Controls for individual differences

Disadvantages
Carry-over effects - second treatment can be affected by first treatment

Order may influence results - which treatment is first can affect the results

Hypotheses

Let D1 , ..., Dn be a small random sample (n <= 30) of the differences in pairs.

H0 : μD = μ0 ,
H1 : μD = μ0 .
Almost always μ0 = 0.

Steps to conduct dependent t-test

Calculate mean difference of the pairwise differences

Calculate standard deviation of the pairwise differences

Calculate t-statistic and compare it with t-critical value

If |t-statistic| > t-critical value, reject H0 .

t-statistic
Under H0

Dˉ − μ0
T = ∼ tn−1
sD / n

Two-sample t-tests 4
 Dˉ is the average of the differences. μ0 is the average of the differences under the null hypothesis, and
sD is the standard deviation of sample differences.

E.g. Evaluate the effect of stretching on height.

You measure the height of a group of people who do not stretch regularly. They then start stretching
regularly for a year. You measure their heights again and take the difference. You want to see if there is a
statistically significant effect, i.e. if the population mean differs from zero.

H0 : μ1 = μ2 ,
H1 : μ1 =
 μ2 .

So we fail to reject the null hypothesis. There is probably no change in height after the additional
stretching.

Confidence interval

A level (1 − α) ∗ 100% confidence interval for the mean difference μD is given by

If the sample size is large, then the confidence interval for the mean difference μD is

Two-sample t-tests 5