Chapter 3

Basic Probability Concepts

Prepared by
Hirbo Shore (MPH, Assistant Professor of
Epidemiology)
School of public-CHMS, HU
Contact: [email protected]

10/17/2019 1
 Basic Probability Concepts
• Probability is the foundation for making inference about the
population based on the sample as representative part of the
population.

• Frequently encountered in everyday communication

• E.g. we may hear a say that a patient has a 50–50 chance of

surviving a certain operation.

• Another physician may say that she is 95 percent certain that a

patient has a particular disease
10/17/2019 2
 Probability

• A measure used to measure the chance of the occurrence of

some event.

• The more likely the event, the closer the number is to one

• The more unlikely the event, the closer the number is to zero.

• An event that cannot occur has a probability of zero, and an

event that is certain to occur has a probability of one.

10/17/2019 3
 Experiment
• A procedure (process) that we perform whose outcomes are
not predictable in advance.
• Example:
– Experiment with coin tossing results in two outcomes, head
or tail, but the outcome is not known until the coin is tossed
or until the experiment is conducted.

10/17/2019 4
 VIEWS OF PROBABILITY

• The concept of objective probability may be categorized

further under the headings of;

1. Objective probability

2. Subjective probability

10/17/2019 5
 Objective probability
• The objective probability may be categorized further under the
headings of
1. Classical probability,
2. the relative frequency concept of probability
• Classical Probability
– developed out of attempts to solve problems related to
games of chance
• Examples: if a fair six-sided die is rolled, the probability that a
1 will be observed is equal to 1/6 and is the same for the other
five faces

10/17/2019 6
 Objective probability,…
• Definition
– If an event can occur in N mutually exclusive and equally
likely ways, and if m of these possess a trait E, the
probability of the occurrence of E is equal to m/N.

– If we read P(E) as “the probability of E,” we may express

𝑚
this definition as P(E)=
𝑁

10/17/2019 7
 Relative Frequency Probability

• If some process is repeated a large number of times, n, and if

some resulting event with the characteristic E occurs m times,
the relative frequency of occurrence of E, m/n, will be
approximately equal to the probability of E.
𝑚
• This definition in compact form written P(E) =
𝑛

10/17/2019 8
 ELEMENTARY PROPERTIES OF PROBABILITY

• 1. A probability value must lie between 0 and 1,

0≤P(E)≤1

– A value 0 means the event can not occur

– A value 1 means the event definitely will occur

– A value of 0.5 means that the probability that the event will
occur is the same as the probability that it will not occur.

10/17/2019 9
 Elementary properties,…
2. The sum of the probabilities of all mutually exclusive
outcome is equal to 1.
P(E1) + P(E2 ) +... + P(En ) = 1
3. For any two events A and B
P(A or B) = P(A) + P(B) ‐ P(A and B)
For two mutually exclusive events A and B,
P(A or B ) = P(A) + P(B)

10/17/2019 10
 Elementary properties,…

4. For any two independent events A and B

P(A and B) = P(A) P(B)

5. If A’ is the complementary event of the event A, then

P(A’) = 1 ‐ P(A).

10/17/2019 11
 Elementary properties,…
Example
• A study investigating the effect of prolonged exposure to
bright light on retina damage in premature infants. Eighteen of
21 premature infants, exposed to bright light developed
retinopathy, while 21 of 39 premature infants exposed to
reduced light level developed retinopathy. For this sample, the
probability of developing retinopathy is

10/17/2019 12
 Elementary properties,…

№𝑜𝑓 𝑖𝑛𝑓𝑎𝑛𝑡𝑠 𝑤𝑖𝑡ℎ 𝑟𝑒𝑡𝑖𝑛𝑜𝑝𝑎𝑡ℎ𝑦

P(retinopathy) =
𝑡𝑜𝑡𝑎𝑙 № 𝑜𝑓 𝑖𝑛𝑡𝑎𝑛𝑡𝑠

18+21
= = 0.65
21+39

10/17/2019 13
 Conditional probability
• In the retinopathy study described in the above example, the
primary concern is comparison of the bright‐light infants with
the reduced‐light infants. We want to know whether the
probability of retinopathy for the bright‐light infants differs
form the probability of retinopathy for the reduced‐light
infants. These probabilities are conditional probabilities,
probabilities based on the knowledge that some event has
occurred.

10/17/2019 14
 Conditional probability,…
• We want to compare the probability of retinopathy, given that
the infant was exposed to bright light, with that the infant was
exposed to reduced light.

• Exposure to bright light and exposure to reduced light are

conditioning events, events we want to take into account when
calculating conditional probabilities.

10/17/2019 15
 Conditional probability,…
• Denoted by P(A/B) or P(Event/conditioning event).

• The formula for calculating a sample conditional probability

№ of event
and conditioning event both occur
is: P(event/conditioning event) = № observations for which
conditioning event occurs

𝑃(𝐴∩𝐵)
P(A/B) = , 𝑖𝑓 𝑃 𝐵 >0
𝑃(𝐵)

10/17/2019 16
 Conditional probability,…
• For the retinopathy data, the conditional probability of
retinopathy, given exposure to bright light, is

• P(Retinopathy/exposure to bright light)=

𝑖𝑛𝑓𝑎𝑛𝑡𝑠 𝑤𝑖𝑡𝑕 𝑟𝑒𝑡𝑖𝑛𝑜𝑝𝑎𝑡𝑕𝑦 𝑒𝑥𝑝𝑜𝑠𝑒𝑑 𝑡𝑜 𝑏𝑟𝑖𝑔𝑕𝑡 𝑙𝑖𝑔𝑕𝑡
𝑖𝑛𝑓𝑎𝑛𝑡𝑠 𝑒𝑥𝑝𝑜𝑠𝑒𝑑 𝑡𝑜 𝑏𝑟𝑖𝑔𝑕𝑡 𝑙𝑖𝑔𝑕𝑡

=18/21=0.86

10/17/2019 17
 Conditional probability,…

• P(Retinopathy/exposure to reduced light) =

No. of infants with retinopathy exposed to reduced light

No. of infants exposed to reduced light

=21/39 = 0.54

• The conditional probabilities suggest that premature infants

exposed to bright light have a higher risk of retinopathy than
premature infants exposed to reduced light.

10/17/2019 18
 Conditional probability,…
• For independent events A and B

P(A/B) = P(A).

• For non independent events A and B

P(A and B) = P(A/B) P(B)

(General Multiplication Rule)

10/17/2019 19
 Probability distribution

• The relationship between the values of a random variable and

the probabilities of their occurrence

• Expressed in the form of a table, graph, or formula

• Provides the clinician and researcher with a powerful tool for

summarizing and describing a set of data and for reaching
conclusions about a population of data on the basis of a sample
of data drawn from the population.

10/17/2019 20
 Probability Distributions for Discrete
Random Variables
• A table, graph, formula, or other device used to specify all
possible values of a discrete random variable along with their
respective probabilities
• shows the probability associated with each value of the
discrete random variable.

• The relative frequency based on the sample data approximates

the probability distribution for a large sample size.

10/17/2019 21
 Probability Distributions,…
• The main properties of a discrete random variable are:
– the probability of every value of a variable lies between 0
and 1

– the sum of all probabilities of the variable is equal to 1

– the probabilities of a discrete random variable are additive

• If we let the discrete probability distribution be represented by

P(x), then P(x)= P(X=x) is the probability of the discrete
random variable X to assume a value x.
10/17/2019 22
 Probability Distributions,…

• The probability distribution of a discrete variable, X, can be

shown for different number of outcome values of the variable.

• Some examples are shown below:

• P(X =x); x =0, 1.
• P(X=x); x = 0, 1, . . . K.
• P(X = x); x =0, 1, . . .∞.

10/17/2019 23
 Probability Distributions,…

• The data on number of daily injury accidents for a period is

used in this example (Leiter and Hamdan 1973). Let us define
the following discrete random variable X = the number of
daily injury accidents in a day.

10/17/2019 24
 Probability Distributions,…
Frequency distribution of number of injury accidents in a day

Possible values of the random variable X are X = 0, 1, 2, 3, 4, 5.

Experiment: Selecting a day at random

10/17/2019 25
 Probability Distributions,…

• (X = 0) = The event that on the selected day there was no accident,

• (X = 1) = The event that on the selected day there was one accident,

• (X = 2) = The event that on the selected day there were 2 accidents,

• (X = 3) = The event that on the selected day there were 3 accidents,

• (X = 4) = The event that on the selected day there were 4 accidents,

• (X = 5) = The event that on the selected day there were 5 accidents.

10/17/2019 26
 Probability Distributions,…

• In general
– (X = x) = the event that on the selected day there were x
accidents.
– Using the relative frequency definition, we know that n =
639.
– The number of elements of the event (X = x) is n(X = x) =
frequency of x.
𝑛 𝑋=𝑥 𝐹𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦
– P(X =x)= =relative frequency =
639 639

10/17/2019 27
 Probability Distributions,…
Probability distribution of number of accidents in a day

What the probability that one accident would

occur?

10/17/2019 28
 Probability Distributions,…
• Solution
P(X=1)=0.3380
What is the probability that two or more accidents?
Solution:
P(X≥2)=P(X=2) + P(X=3) + P(X=4) + P(X=5)
= 0.1140 + 0.1469 + 0.0219 + 0.0016
= 0.2144

10/17/2019 29
 Probability Distributions,…
0.45
0.4
0.35
0.3
P(X=x)

0.25
0.2
0.15
0.1
0.05
0
0 1 2 3 4 5
X=x
The graphical presentation of this probability distribution

10/17/2019 30
 Expected Value and Variance of a Discrete
Random Variable
• The population mean and variance are defined using
corresponding probabilities of a variable

• The population mean is defined as the expected value of a

variable and it can be expressed as follows:

– The population mean or expected value of a discrete

random variable X is denoted by µ or µx and it is defined
by

10/17/2019 31
 E(X)=μ= 𝑥 𝑥𝑃(𝑋 = 𝑥)
• Similarly, the population variance of a discrete random
variable X is denoted by 𝜎2 or 𝜎x2 and it is defined by
Var(X)=E(X-μ)2 =σ2 = 𝑥 (X−μ)2P(X=x)
• Example: calculate the μ and the 𝜎2 of the discrete random
variable X whose probability distribution is given by the
following table

10/17/2019 32
• Solution
X P(X=x) xP(=x) (x-μ) (x-μ)2 (x-μ)2 P(X=x)
0 0.05 0 -1.9 3.61 0.1805
1 0.25 0.25 -0.9 0.81 0.2025
2 0.45 0.90 0.1 0.01 0.0045
3 0.25 0.75 1.1 1.21 0.3025
μ= σ2 = 2P(X=x)=0.69
Total 𝑥 𝑥𝑃(𝑋 = 𝑥) 𝑥 (X−μ)

10/17/2019 33
 Cumulative Distributions

• The cumulative probability distribution for the discrete

variable whose probability distribution

• obtained by successively adding the probabilities, P (X =xi)

• The cumulative probability for xi is written as F(xi)=P(X≤ xi)

• It gives the probability that X is less than or equal to a

specified value, xi

10/17/2019 34
 Cumulative Distributions,…

Cumulative probability distribution

10/17/2019 35
 THE BINOMIAL DISTRIBUTION
• Widely encountered probability distributions in applied
statistics

• Derived from a process known as a Bernoulli trial

• Arises from trial that result in only one of two mutually

exclusive outcomes, such as dead or alive, sick or well, full-
term or premature
• Outcomes which are termed traditionally as success or failure.

10/17/2019 36
 The Bernoulli Process
1. Each trial results in one of two possible, mutually exclusive,
outcomes. One of the possible outcomes is denoted
(arbitrarily) as a success, and the other is denoted a failure.

2. The probability of a success, denoted by p, remains constant

from trial to trial. The probability of a failure, 1- p, is denoted
by q.

3. The trials are independent; that is, the outcome of any

particular trial is not affected by the outcome of any other
trial.
10/17/2019 37
 The Bernoulli Process,…

4. The probabilities of success or failure are

complementary to each other so that if the probability of
success is denoted by p then the probability of failure is
1-P, where p satisfies 0≤p ≤ 1.

10/17/2019 38
 Binomial Distribution,…
• example; if all birth records for a calendar year shows that
85.8% of the pregnancies had delivery in week 37 or later.

• The 85.8% interpreted as the probability of a recorded birth in

week 37 or later

• If we randomly select five birth records from this population,

what is the probability that exactly three of the records will be
for full-term births?

10/17/2019 39
 Binomial Distribution,…

• Let us designate the occurrence of a record for a full-term birth

(F) as a “success,” and hasten to add that a premature birth (P)
is not a failure

• It will also be convenient to assign the number 1 to a success

(record for a full-term birth) and the number 0 to a failure
(record of a premature birth).

10/17/2019 40
 Binomial Distribution,…

• Suppose the five birth records selected resulted in this

sequence of fullterm births

FPFFP
• In coded form we would write this as
10110
P(1, 0, 1, 1, 0,) =pqppq=q2p3

10/17/2019 41
 Binomial Distribution,…
• Three successes and two failures could occur in any one of the
following additional sequences as well:

From the addition rule we know that this

probability is equal to the sum of the
individual probabilities. In the present
example we need to sum the 10q2p3’s or,
equivalently, multiply q2p3 by 10.

10/17/2019 42
 Binomial Distribution,…

• Answer for original question is

• Since in the population, p=0.858;

q=(1-p)=(1- 0.858)=0.142

10q2p3 =10(0.142)2(0.858)3
=10 (0.0202)(0.6316)
= 0.1276

10/17/2019 43
 Combinations
• A combination of n objects taken x at a time is an unordered
subset of x of the n objects
• Combination is used in large sample procedures
• The number of combinations of n objects that can be formed
by taking x of them at a time is given by
𝑛!
n𝐶 x= 𝑥! 𝑛−𝑥 !
• where x!, read x factorial, is the product of all the whole
numbers from x down to 1. That is,
• x! =x(x-1)(x-2)…(1). We note that, by definition, 0!=1.

10/17/2019 44
 Binomial Distribution,…

• Let us return to our example in which we have a sample of

n=5 birth records and we are interested in finding the
probability that three of them will be for full-term births

5! 5𝑥4𝑥3𝑥2𝑥1 120
5𝐶 3= 3! 5−3 !
=
(3𝑥2𝑥1) 2𝑥1
=
12
= 10

10/17/2019 45
 Binomial Distribution,…

• In our example we let x =3, the number of successes, so that n-

x = 2, the number of failures. We then may write the
probability of obtaining exactly x successes in n trials as

f(x)= n𝐶 x 𝑞 𝑛−𝑥 𝑝 𝑥 =n𝐶x 𝑝 𝑥 𝑞 𝑛−𝑥 , for x=0, 1, 2, …,n

• The Binomial Parameters
– binomial distribution has two parameters, n and p
– The μ=np,
– σ2=np(1-p)
=npq
10/17/2019 46
 Poisson Distribution

• Used to model a discrete random variable representing the

number of occurrences or counts of some random events in an
interval of time or space (or some volume of matter)

• The possible values of X = x are x = 0, 1, 2, 3,…

• The discrete random variable, X, is said to have a Poisson

distribution with parameter (mean) k if the probability

𝑒 −λ λ𝑥
distribution of X is given by f(x) =
𝑥!

10/17/2019 47
 Poisson Distribution,…
• where e = 2.71828 (the natural number).

• λ (lambda) is the parameter of the distribution and is the

average number of occurrences of the random event in the
interval

• The Poisson Process

– The occurrences of the events are independent

– The probability of the single occurrence of the event in a
given interval is proportional to the length of the interval

10/17/2019 48
 Poisson Distribution,…
– In any infinitesimally small portion of the interval, the
probability of more than one occurrence of the event is
negligible

• EXAMPLE
– In a study of drug-induced anaphylaxis among patients
taking rocuronium bromide as part of their anesthesia the
occurrence of anaphylaxis followed a Poisson model with
λ=12 incidents per year in Norway

10/17/2019 49
 Poisson Distribution,…

– Find the probability that in the next year, among patients

receiving rocuronium, exactly three will experience
anaphylaxis

𝑒 −12 123
f(x=3) = = 0.00177
3!

• What is the probability that at least three patients in the next

year will experience anaphylaxis if rocuronium is administered
with anesthesia?

10/17/2019 50
 Poisson Distribution
• Example: Suppose that the number of accidents per day in a
city has a Poisson distribution with average 2 accidents.
1. What is the probability that in a day
I. the number of accidents will be 5,
II. the number of accidents will be less than 2.
2. What is the probability that there will be six accidents
in 2 days?
3. What is the probability that there will be no accidents
in an hour?

10/17/2019 51
 Poisson Distribution,…

𝑒 −2 25
1. P (X =5) = = 0.036089
5!

2. P(X<2)=P(X=0) + P(X=1) =>

𝑒 −2 20 𝑒 −2 21
= + =0.135335 + 0. 270670= 0.406005.
0! 1!

10/17/2019 52
 Probability Distributions of Continuous data
• A nonnegative function f(x) is called a probability distribution
(sometimes called a probability density function) of the
continuous random variable X if the total area bounded by its
curve and the x -axis is equal to 1 and if the subarea under the
curve bounded by the curve, the x -axis, and perpendiculars
erected at any two points a and b give the probability that X is
between the points a and b.

10/17/2019 53
 Probability Distributions of Continuous data…

Graph of a continuous distribution showing area between a

and b.

10/17/2019 54
 NORMAL DISTRIBUTION
• Frequently called the Gaussian distribution in recognition of
his contributions

• The normal density is given by

1 − 𝑥−𝜇 2
/2𝜎2
f(x)= 𝑒 , −∞ < 𝑥 < ∞;
2𝜋𝜎

where (e = 2.71828) and (π = 3.14159).

• The parameters of the distribution are the µ and the σ2
X ~N (μ,σ2).

10/17/2019 55
 Normal Distribution,…
• The density function of X, f(x), is a bell-shaped curve
– The highest point of the curve of f(x) is at the mean μ.
Hence, the mode = mean = μ.
– The curve of f(x) is symmetric about the mean μ.

– In other words, mean = mode = median

– The area under the curve is 1

10/17/2019 56
 Standard Normal Distribution
• the standard normal distribution with mean µ = 0 and variance
σ2 =1
• Denoted by Normal (0,1) or N(0,1).
• The standard normal random variable is denoted by Z, and we
write Z~N(0,1)
𝑥 −𝜇
Z=
𝜎
• The equation for the standard normal distribution is written

1 −𝑧2/2
Z= 𝑒 , −∞ < 𝑧 < ∞
2𝜋

10/17/2019 57
 Standard Normal Distribution,…
The standard normal distribution

The z-transformation is useful in application of normal distribution

10/17/2019 58
 Standard Normal Distribution,…

• Z-transformation that yields a value of Z=1 indicates that the

value of x used in the transformation is 1 standard deviation
above 0.

• A value of Z = -1 indicates that the value of x used in the

transformation is 1 standard deviation below 0.

10/17/2019 59
 Standard Normal Distribution,…
• Example;
– What is the probability that a z picked at random from the
population of z’s will have a value between 2:55 and
þ2:55?
answer:
P(-2.55<z<2.55)=0.9946- 0.0054
=0.9892

10/17/2019 60
 Standard Normal Distribution,…

10/17/2019 61
 Application of normal distribution
• Normal distribution is not a law that is adhered to by all
measurable characteristics occurring in nature
• However, many of these characteristics are approximately
normally distributed
• Used to model the distribution of many variables that are of
interest
• Allows us to make useful probability statements about some
variables conveniently than would be the case if some more
complicated model had to be used

10/17/2019 62
 Application,…
• Example:
– Let us consider weight of women in reproductive age
follows a normal distribution with mean 49 kg and variance
25 kg2
a. Find the probability that a randomly chosen woman in
her reproductive age has weight less than 45 kg.
b. What is the percentage of women having weight less
than 45 kg?
c. In a population of 20,000 women of reproductive age,
how many would you expect to have weight less than
45 kg?
10/17/2019 63
• Solution Here the random variable, X = weight of women in
reproductive age, population mean = 49 kg, population
variance= σ2 = 25 kg2, population standard deviation = σ = 5
kg. Hence, X~Normal (49,25).
a. The probability that a randomly chosen woman in
reproductive age has weight less than 45 kg is P(X<45)

10/17/2019 64
 Application,…

– The percentage of women of reproductive age who have

weight less than 45 kg is P(x<45) x100% = 0.2119 x100%
= 21.19%
• In a population of 20,000 women of reproductive age, we
would expect that the number of women with weight less than
45 kg is P(X <45)x 20,000 =0.2119 x20,000 = 4238.

10/17/2019 65
 !!
THANK YOU FOR YOUR ATTENTION!!

10/17/2019 66