Random Variables and Probability Distributions

From "what's the chance of this one thing?" to describing every possible outcome at once: random variables, the PMF, PDF and CDF, and the expected value that every casino is built on.

PMF chance of each exact value PDF probability is area under a curve CDF running total: P(X ≤ x) 0 1 2 3 4 5 area = probability 1 PMF chance of each exact value PDF probability is area under a curve CDF running total: P(X ≤ x) 0 1 2 3 4 5 area = probability 1
The three tools of today's article. A PMF gives the probability of each exact value. A PDF spreads probability along a curve, where areas are probabilities. A CDF keeps a running total from left to right.

On Day 7 we computed the probability of single events: rolling a 5, a transaction being fraud, an email being spam. But most questions in data science are about a whole range of possible values. How many support tickets will arrive in the next hour? How long will a user browse before choosing something? How much will the next customer spend? Answering those needs two new ideas: random variables and probability distributions.

What is a random variable?

A random variable is a rule that turns each outcome of a random experiment into a number. That's it. Once outcomes are numbers, you can average them, measure their spread and draw them, with every tool from Part I.

Take Day 7's experiment of flipping a coin twice. The outcomes are HH, HT, TH and TT, which aren't numbers. Define X as "the number of heads". Now each outcome has a number: HH gives 2, HT and TH give 1, TT gives 0.

Outcome X = number of heads HH HT TH TT 2 1 0 Its distribution (PMF) 1/4 0 1/2 1 1/4 2 Outcome X = number of heads HH HT TH TT 2 1 0 Its distribution (PMF) 1/4 0 1/2 1 1/4 2
A random variable is a function from outcomes to numbers. Two different outcomes, HT and TH, both map to 1, which is why X = 1 is twice as likely as X = 0 or X = 2.

Because two of the four equally likely outcomes give X = 1, P(X = 1) = 2/4 = 1/2, while P(X = 0) and P(X = 2) are 1/4 each. That list of values and their probabilities is the random variable's distribution, which is the main subject of this article.

Notation

Random variables get capital letters, like X or Y. A particular value gets a lowercase letter, like x. So P(X = x) reads "the probability that the random variable X takes the value x", and P(X = 1) = 0.5 is a specific example.

Discrete and continuous random variables

Random variables come in two main kinds, and the kind decides which tools you use.

A quick test: if you'd answer "how many?", it's usually discrete; if you'd answer "how much?" or "how long?", it's usually continuous. (A few variables are a mix of both. Daily rainfall is exactly 0 on many days but can be any positive amount on the rest. They're less common, and the ideas below still apply in pieces.)

The difference that changes everything

For a discrete variable you can list every value and ask "what's the probability of exactly this one?" For a continuous variable there are infinitely many possible values, and the probability of hitting any single exact value is zero. You can only ask about ranges. Hold on to this: it explains why discrete and continuous variables need different tools.

What is a probability distribution?

A probability distribution is a complete description of a random variable: every value it can take, and how likely each one is. The simplest example is one roll of a fair die. X is the face shown, and each face is equally likely:

x123456Total
P(X = x)1/61/61/61/61/61/61

Every valid distribution follows the two rules from Day 7: each probability is between 0 and 1, and all the probabilities add up to exactly 1, because one of the values must happen.

How you write a distribution down depends on the kind of variable. Discrete variables use a probability mass function (PMF). Continuous variables use a probability density function (PDF). Both can be summarised by a cumulative distribution function (CDF). Let's take them in turn.

The probability mass function (PMF)

For a discrete random variable, the PMF gives the probability of each possible value:

p(x) = P(X = x)
Rules: 0 ≤ p(x) ≤ 1 for every x, and the p(x) values add up to 1.

Think of it as a lookup table: give it a value, and it gives you that value's probability.

Example: support tickets per hour

Suppose a support team looks back over a year of peak-time hours and finds this distribution for X, the number of tickets that arrive in an hour:

x (tickets)012345
P(X = x)0.050.150.300.250.150.10

The probabilities add up to 1, so it's a valid PMF. Now questions become lookups and sums. The chance of exactly 2 tickets is P(X = 2) = 0.30. The chance of more than 3 is P(X = 4) + P(X = 5) = 0.25.

0.05 0 0.15 1 0.30 2 0.25 3 0.15 4 0.10 5 support tickets in an hour (x) P(X > 3) = 0.15 + 0.10 = 0.25 0.05 0 0.15 1 0.30 2 0.25 3 0.15 4 0.10 5 support tickets in an hour (x) P(X > 3) = 0.15 + 0.10 = 0.25
The support-ticket PMF. Each bar's height is a probability. "More than 3" means adding the two highlighted bars.
Pythonimport numpy as np x = np.array([0, 1, 2, 3, 4, 5]) p = np.array([0.05, 0.15, 0.30, 0.25, 0.15, 0.10]) print(f"Sum of probabilities: {p.sum():.2f}") # must be 1 print(f"P(X = 2) = {p[x == 2][0]:.2f}") print(f"P(X > 3) = {p[x > 3].sum():.2f}") print(f"P(X <= 1) = {p[x <= 1].sum():.2f}")
OutputSum of probabilities: 1.00 P(X = 2) = 0.30 P(X > 3) = 0.25 P(X <= 1) = 0.20

A PMF from real data

In practice you often estimate a PMF from data by counting how often each value appears. The tips dataset that ships with seaborn records 244 restaurant bills, collected by one waiter over a few months. One column is the size of the dining party:

Pythonimport seaborn as sns tips = sns.load_dataset("tips") pmf_size = tips["size"].value_counts(normalize=True).sort_index() print(pmf_size.round(3))
Outputsize 1 0.016 2 0.639 3 0.156 4 0.152 5 0.020 6 0.016 Name: proportion, dtype: float64

That's an empirical PMF: almost two in three parties (63.9%) were a party of two, and parties of five or six were rare. With more data, these proportions settle closer to the true probabilities for that restaurant, the law of large numbers from Day 7 again.

Expected value and variance

On Day 3 we summarised data with a mean, and on Day 4 with a variance and standard deviation. Random variables have exactly the same summaries.

Expected value: the long-run average

The expected value of X, written E[X] or μ, is the average you'd get if you could repeat the experiment forever. For a discrete variable, multiply each value by its probability and add:

E[X] = ∑ x · P(X = x)
For the support tickets: 0(0.05) + 1(0.15) + 2(0.30) + 3(0.25) + 4(0.15) + 5(0.10) = 2.6
0.05 0 0.15 1 0.30 2 0.25 3 0.15 4 0.10 5 E[X] = 2.6: the balance point support tickets in an hour (x) 0.05 0 0.15 1 0.30 2 0.25 3 0.15 4 0.10 5 E[X] = 2.6: the balance point support tickets in an hour (x)
Picture the bars as weights on a beam. The expected value is where the beam balances: 2.6 tickets per hour.

Notice that 2.6 isn't a value X can ever take: you can't receive 2.6 tickets in an hour. That's normal. A fair die's expected value is 3.5, which no roll ever shows. The expected value is a long-run average, not a prediction for any single hour. Over 100 peak hours, though, you should expect about 260 tickets, and that's exactly the kind of number you staff a team around.

Variance: the spread around it

Variance works just like Day 4's, with probabilities as the weights:

Var(X) = ∑ (x − μ)2 · P(X = x)    SD = √Var(X)
Pythonmean = (x * p).sum() variance = ((x - mean) ** 2 * p).sum() print(f"E[X] = {mean:.2f} tickets per hour") print(f"Var(X) = {variance:.2f}") print(f"SD = {np.sqrt(variance):.2f} tickets")
OutputE[X] = 2.60 tickets per hour Var(X) = 1.74 SD = 1.32 tickets

A typical hour sees 2.6 tickets, give or take about 1.3. You'll see E[X] and Var(X) for every named distribution on Day 9 and Day 10.

Real case: every casino runs on expected value

An American roulette wheel has 38 pockets: the numbers 1 to 36 plus 0 and 00. Bet $1 on a single number and, if it comes up, you're paid $35 and keep your dollar. Otherwise you lose the dollar. So:

E[winnings] = 35 × (1/38) + (−1) × (37/38) = −2/38 ≈ −$0.0526

On average, every dollar bet loses about 5.3 cents. That's the "house edge". On a European wheel, with a single zero and 37 pockets, it's 1/37, about 2.7 cents. Any single spin can go either way, but over millions of bets the casino's average take settles right on the expected value. This is the law of large numbers working for the house.

Pythonrng = np.random.default_rng(7) spins = rng.integers(0, 38, size=1_000_000) # pockets 0..37; say pocket 17 is our number winnings = np.where(spins == 17, 35, -1) print(f"Exact expected value per $1 bet: {-2/38:.4f}") print(f"Average over 1,000,000 spins: {winnings.mean():.4f}")
OutputExact expected value per $1 bet: -0.0526 Average over 1,000,000 spins: -0.0504

The probability density function (PDF)

Now try the PMF idea on a continuous variable, such as how long a user browses before picking something to watch. What's the probability they browse for exactly 62.000000... seconds? Zero. There are infinitely many possible times, so any single one has probability 0. A bar chart would need infinitely many bars, all of height zero.

The fix is to spread probability along a curve, the probability density function, and to measure probability as area under it:

P(a ≤ X ≤ b) = area under f(x) between a and b
Rules: f(x) ≥ 0 everywhere, and the total area under the whole curve is 1.

The height of the curve, f(x), is called the density. It tells you where values are concentrated, but density is not a probability. It can even be bigger than 1. If a quantity is spread evenly between 0 and 0.5, its density is 2 across that range, because 2 × 0.5 = 1 is the total area. Only areas are probabilities.

Example: how long do viewers browse?

Real case: Netflix's 60 to 90 seconds

In a 2015 paper on Netflix's recommender system, Netflix's Carlos Gomez-Uribe and Neil Hunt wrote that their consumer research suggested "a typical Netflix member loses interest after perhaps 60 to 90 seconds of choosing". That makes browsing time a quantity worth understanding.

Let's model it. Suppose browsing time X, in seconds, follows an exponential distribution with an average of 90 seconds: most people choose quickly, and a few browse for a long time. (This is an illustrative model, not Netflix's data. We'll meet the exponential distribution properly on Day 10.) SciPy can compute the areas for us:

Pythonfrom scipy import stats browse = stats.expon(scale=90) # exponential distribution with mean 90 seconds p_30_to_120 = browse.cdf(120) - browse.cdf(30) print(f"P(30 to 120 seconds): {p_30_to_120:.3f}") print(f"P(under 60 seconds): {browse.cdf(60):.3f}") print(f"P(over 3 minutes): {browse.sf(180):.3f}") print(f"Density at 0 seconds: {browse.pdf(0):.4f} (a height, not a probability)")
OutputP(30 to 120 seconds): 0.453 P(under 60 seconds): 0.487 P(over 3 minutes): 0.135 Density at 0 seconds: 0.0111 (a height, not a probability)
area = 0.453 P(30 ≤ X ≤ 120) 0 60 120 180 240 300 360 0.004 0.008 browsing time in seconds density f(x) the height is density, not probability area = 0.453 P(30 ≤ X ≤ 120) 0 60 120 180 240 300 360 0.004 0.008 browsing time in seconds density f(x) the height is density, not probability
The shaded area, not the height of the curve, is the probability: 45.3% of browsing sessions last between 30 seconds and 2 minutes under this model.

Under this model, almost half of users (48.7%) choose within a minute, and 13.5% are still browsing after three minutes. That last group is the one at risk of giving up.

Where do the areas come from? SciPy's cdf method, which we'll meet in a moment, gives the area to the left of a point, so the area between 30 and 120 is the area left of 120 minus the area left of 30. And sf, the "survival function", is simply 1 minus the CDF: the area to the right.

The cumulative distribution function (CDF)

Many practical questions are about thresholds rather than exact values. What fraction of customers cancel within 30 days? What share of requests finish in under 200 milliseconds? What's the chance of 2 tickets or fewer? The CDF answers all of them:

F(x) = P(X ≤ x)
The probability that X is at most x. The same definition works for discrete and continuous variables.

For a discrete variable, F(x) adds up the PMF from the left up to x. For a continuous variable, it's the area under the PDF to the left of x. Either way, every CDF has the same basic behaviour: it starts at 0, never goes down, and ends at 1. For a discrete variable it climbs in steps, one step per possible value; for a continuous variable it climbs smoothly.

Building a CDF from a PMF

x (tickets)P(X = x)F(x) = P(X ≤ x)
00.050.05
10.150.05 + 0.15 = 0.20
20.300.20 + 0.30 = 0.50
30.250.50 + 0.25 = 0.75
40.150.75 + 0.15 = 0.90
50.100.90 + 0.10 = 1.00
PMF: P(X = x) 0.05 0 0.15 1 0.30 2 0.25 3 0.15 4 0.10 5 CDF: F(x) = P(X ≤ x) 0.05 0 0.20 1 0.50 2 0.75 3 0.90 4 1.00 5 PMF: P(X = x) 0.05 0 0.15 1 0.30 2 0.25 3 0.15 4 0.10 5 CDF: F(x) = P(X ≤ x) 0.05 0 0.20 1 0.50 2 0.75 3 0.90 4 1.00 5
The same distribution two ways. Each step in the CDF is exactly as tall as the PMF bar at that value. By x = 2, half of all the probability has been collected.
Pythoncdf = np.cumsum(p) # a running total turns the PMF into the CDF print(f"P(X <= 2) = {cdf[2]:.2f}") print(f"P(X > 3) = {1 - cdf[3]:.2f}") print(f"P(1 <= X <= 3) = {cdf[3] - cdf[0]:.2f}")
OutputP(X <= 2) = 0.50 P(X > 3) = 0.25 P(1 <= X <= 3) = 0.70

The CDF turns questions about ranges into subtraction. Two details are worth knowing:

Running the CDF backwards: percentiles

The CDF answers "what's the probability of being at or below x?" Turn it around and you get "below which value do p% of outcomes fall?" That's a percentile (or quantile), and the function that computes it is the inverse CDF. The median is the 50th percentile, and the quartiles from Day 4 are the 25th and 75th.

Pythonprint(f"Median browsing time: {browse.ppf(0.5):.1f} seconds") print(f"90th percentile: {browse.ppf(0.9):.1f} seconds")
OutputMedian browsing time: 62.4 seconds 90th percentile: 207.2 seconds

In SciPy the inverse CDF is called ppf, the "percent point function". Half of users choose within 62.4 seconds, and 90% within 207 seconds. Notice that the median (62.4) is below the mean (90): this distribution is right-skewed, exactly the pattern from Day 3.

PDF: density CDF: P(X ≤ x) 0.5 62 s 0.9 207 s 0 120 240 360 0 120 240 360 1 PDF: density CDF: P(X ≤ x) 0.5 62 s 0.9 207 s 0 120 240 360 0 120 240 360 1
The same continuous distribution as a density and as a CDF. Where the density is tall, near 0, the CDF climbs fastest. Reading across from 0.5 and 0.9 on the CDF gives the median and the 90th percentile.
Real case: growth charts are CDFs

When a doctor says a baby is "at the 75th percentile for length", they're reading a CDF: 75% of healthy babies of that age and sex are shorter. In the US, the Centers for Disease Control and Prevention recommends the World Health Organization's growth standards for children under two, and uses the 2nd and 98th percentiles as cut-offs that flag possible problems, such as unusually low weight for length. Percentiles turn a raw measurement into "how unusual is this?", which is exactly what a CDF is for.

An empirical CDF from real data

You can also estimate a CDF straight from data: F(x) is just the fraction of observations at or below x. Back to the restaurant bills:

Pythonbills = tips["total_bill"] print(f"Share of bills at or below $20: {(bills <= 20).mean():.1%}") print(f"Median bill: ${bills.median():.2f}") print(f"90th percentile bill: ${bills.quantile(0.9):.2f}")
OutputShare of bills at or below $20: 60.2% Median bill: $17.80 90th percentile bill: $32.24

60% of the bills were $20 or less, half were under $17.80, and only one in ten topped $32.24. No formula needed: the data is its own distribution.

PMF, PDF or CDF?

ToolForAnswersWatch out for
PMF, p(x)Discrete variables"What's the chance of exactly x?"The values must add up to 1.
PDF, f(x)Continuous variables"Where are values concentrated?" Probabilities come from areas.The height is density, not probability, and can exceed 1.
CDF, F(x)Both"What's the chance of x or less?" Plus ranges by subtraction, and percentiles by running it backwards.For discrete variables, < and ≤ give different answers.

Key takeaways

Quick check

Six questions to test whether this stuck. The answers are one click away.

Is "the number of typos in a blog post" discrete or continuous? What about "the time between two typos"?

The number of typos is discrete: 3 or 4, never 3.5. The time between them is continuous: it could be 12.3 seconds or 12.34 seconds, with no gaps between possible values.

A PMF has P(X = 1) = 0.2, P(X = 2) = 0.5, and P(X = 3) unknown. These are the only possible values. What is P(X = 3)?

0.3. A PMF must add up to 1, so P(X = 3) = 1 − 0.2 − 0.5 = 0.3.

For that same PMF, what is the expected value E[X]?

E[X] = 1(0.2) + 2(0.5) + 3(0.3) = 0.2 + 1.0 + 0.9 = 2.1.

Why is it meaningless to ask "what's the probability a session lasts exactly 47.000 minutes"?

Session length is continuous, so any single exact value has probability 0. A question with a real answer is "what's the probability the session lasts between 46 and 48 minutes?", which is the area under the PDF over that range.

A PDF reaches a height of 3 at one point. Is something wrong?

No. Density isn't probability, so it can exceed 1, as long as the total area under the curve is 1. A variable spread evenly between 0 and 1/3 has a density of exactly 3.

A CDF has F(10) = 0.4 and F(20) = 0.75. What is P(10 < X ≤ 20)?

0.35. F(20) − F(10) = 0.75 − 0.4. Once you have a CDF, the probability of any range is just a subtraction.

Next up, Day 9 Discrete Probability Distributions

Some PMFs turn up so often that they have names: Bernoulli, binomial, geometric and Poisson. Next time: how to recognise each one, and what its expected value and variance tell you.