Proability and Statistics
MA20205
Bibhas Adhikari
Autumn 2022-23, IIT Kharagpur
Lecture 2
August 16, 2022
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 1 / 15
Probability
□ Probability theory - Kolmogorov
□ Probability theory 7→ Mathematical statistics
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 2 / 15
Probability
□ Probability theory - Kolmogorov
□ Probability theory 7→ Mathematical statistics
Sample space
▶ Discrete sample space - countable sample points
▶ Continuous sample space - uncountable sample points
□ Example: roll a pair of dice:
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 2 / 15
Probability
□ Probability theory - Kolmogorov
□ Probability theory 7→ Mathematical statistics
Sample space
▶ Discrete sample space - countable sample points
▶ Continuous sample space - uncountable sample points
□ Example: roll a pair of dice:
S = {(x, y ) : 1 ≤ x ≤ 6, 1 ≤ y ≤ 6}
▶ Events: sum of the outcomes is equal to 8
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 2 / 15
Probability
σ-field
A set A ⊆ S is an event if A ∈ F ⊂ P(S), power set of S such that
S ∈F
if A ∈ F then Ac ∈ F
S∞
if Aj ∈ F, j ≥ 1 then j=1 Aj ∈F
Question: Why do we need the notion of σ-field?
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 3 / 15
Probability
Question: Probability of what?
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 4 / 15
Probability
Question: Probability of what?
Frequentist Interpretation of Probability and classical definition
n(A) #A
P(A) = =
n(S) #S
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 4 / 15
Probability
Question: Probability of what?
Frequentist Interpretation of Probability and classical definition
n(A) #A
P(A) = =
n(S) #S
Probability measure (axiomatic definition)
Probability is function P : F → [0 1] - it assigns nonnegative numbers to
events such that
P(S) = 1
(addition rule)
∞ ∞
!
[ X
P Ak = P(Ak )
k=1 k=1
if A1 , A2 , . . . , Ak , . . . are disjoint events of S
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 4 / 15
Probability
Properties of probability
P(∅) = 0
For any finite collection of mutually exclusive events:
{A1 , A1 , . . . , An } with Ai ∩ Aj = ∅ if i ̸= j then
n n
!
[ X
P Ak = P(Ak )
k=1 k=1
For a discrete sample space S = {s1 , s2 , . . .},
∞
X
P(S) = P({sk })
k=1
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 5 / 15
Probability
Properties of probability
P(∅) = 0
For any finite collection of mutually exclusive events:
{A1 , A1 , . . . , An } with Ai ∩ Aj = ∅ if i ̸= j then
n n
!
[ X
P Ak = P(Ak )
k=1 k=1
For a discrete sample space S = {s1 , s2 , . . .},
∞
X
P(S) = P({sk })
k=1
Question: Does P(A) = 1 imply A = S?
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 5 / 15
Probability
Question: Is the axiomatic definition a more general definition than the
classical one?
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 6 / 15
Probability
Question: Is the axiomatic definition a more general definition than the
classical one?
Derive: for equally likely sample points,
#A
P(A) =
#S
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 6 / 15
Probability
Some more properties of probability measure (use Venn diagram)
P(Ac ) = 1 − P(A)
if A ⊆ B ⊆ S then P(A) ≤ P(B)
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Let A1 ⊆ A2 . Then
P(A2 \ A1 ) = P(A2 ) − P(A1 )
Problems: .....
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 7 / 15
Probability
Question: For a finite sample space, how do we define probability of A
given that B has happened?
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 8 / 15
Probability
Question: For a finite sample space, how do we define probability of A
given that B has happened?
number of elements of A ∩ B
P(A given B) =
number of elements of B
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 8 / 15
Probability
Conditional probability
For both discrete and continuous sample space:
P(A ∩ B)
P(A|B) =
P(B)
provided P(B) > 0.
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 9 / 15
Probability
Conditional probability
For both discrete and continuous sample space:
P(A ∩ B)
P(A|B) =
P(B)
provided P(B) > 0.
Conditional probability measure:
P(A|B) ≥ 0 for all event A
P(B|B) = 1
(addition rule)
∞ ∞
!
[ X
P Ak |B = P(Ak |B)
k=1 k=1
for mutually exclusive events A1 , A2 , . . . , Ak , . . .
Problems...
Bibhas Adhikari (Autumn 2022-23, IIT Kharagpur) Proability and Statistics Lecture 2 August 16, 2022 9 / 15
