Discrete Probability Distributions

Bernoulli, binomial, geometric and Poisson: four named patterns that turn "build a probability model from scratch" into "recognise the pattern, check its assumptions, and plug in the numbers".

Bernoulli(0.12) one yes/no trial 0 1 Binomial(200, 0.08) successes in n trials 0 10 20 30 Geometric(0.15) trials until the first success 1 10 20 Poisson(4.5) events in a fixed window 0 5 10 Bernoulli(0.12) one yes/no trial 0 1 Binomial(200, 0.08) successes in n trials 0 10 20 30 Geometric(0.15) trials until the first success 1 10 20 Poisson(4.5) events in a fixed window 0 5 10
The four distributions in this article, each drawn from one of its worked examples. Same idea as Day 8's PMFs, but each shape comes from a formula with just one or two settings.

On Day 8 we built PMFs by hand from tables of data, and met the expected value E[X] and variance Var(X). Some PMF shapes turn up so often, in so many unrelated problems, that they have names and ready-made formulas. Once you recognise which one fits, you don't need to build anything from scratch: you set one or two numbers, called parameters, and the formula does the rest.

Here are four questions a data analyst might face in a single week. Notice what they share, and how they differ:

  1. A user gets a push notification. Do they open it or not?
  2. 200 users sign up for a free plan, each with an 8% chance of upgrading to paid. How many will upgrade?
  3. A fraud analyst reviews flagged transactions one at a time, and each has a 15% chance of being real fraud. How many will she review before finding the first real case?
  4. A website logs 4.5 errors per hour on average. How many will it log in the next hour?

All four count something, so all four are discrete. But the process underneath is different each time: one yes-or-no event, a count of successes in a fixed number of tries, a wait for the first success, and a count of events arriving at a rate. Each process has its own distribution.

1. The Bernoulli distribution

One trial, two outcomes. Did the user open the notification? Is this email spam? Did the customer cancel? A single experiment with exactly two possible outcomes is a Bernoulli trial, named after the 17th-century Swiss mathematician Jacob Bernoulli. We code the outcomes as numbers: X = 1 for "success" (whatever outcome you're counting) and X = 0 for "failure".

P(X = 1) = p    P(X = 0) = 1 − p
One parameter: p, the probability of success. E[X] = p and Var(X) = p(1 − p).

The variance formula is worth a moment. p(1 − p) is largest at p = 0.5, where it equals 0.25, and shrinks towards 0 as p approaches 0 or 1. That matches intuition: a 50/50 outcome is as unpredictable as it gets, while a 99% outcome is almost a sure thing.

Worked example: push notifications

Suppose past data says 12% of notifications get opened. Then X ~ Bernoulli(0.12), read "X follows a Bernoulli distribution with p = 0.12".

Pythonfrom scipy import stats notification = stats.bernoulli(0.12) print(f"P(opened) = {notification.pmf(1):.2f}") print(f"P(not opened) = {notification.pmf(0):.2f}") print(f"E[X] = {notification.mean():.2f}") print(f"Var(X) = {notification.var():.4f}")
OutputP(opened) = 0.12 P(not opened) = 0.88 E[X] = 0.12 Var(X) = 0.1056
The building block of classification

A binary classifier, such as logistic regression predicting whether a customer will cancel, outputs a probability. When a model says "0.87 chance this customer cancels", it's estimating the p of a Bernoulli trial for that customer. Logistic regression is built directly on this idea, and the Bernoulli distribution is the foundation of the binomial distribution, which comes next.

2. The binomial distribution

Repeat a Bernoulli trial n times and count the successes. Send 10,000 emails and count the opens. Show an offer to 500 users and count the conversions. If the setup meets four conditions, the count follows a binomial distribution:

Where the formula comes from

Show a free trial to 3 users, each with a 50% chance of converting. What's the chance exactly 2 convert? There are three ways it can happen:

User 1User 2User 3Probability
convertsconvertsdoesn't0.5 × 0.5 × 0.5 = 0.125
convertsdoesn'tconverts0.125
doesn'tconvertsconverts0.125

So P(X = 2) = 3 × 0.125 = 0.375. The "3" is the number of ways to choose which 2 of the 3 users convert, which is the combination C(3, 2) from Day 7. The 0.125 is the probability of any one of those arrangements. That's the whole formula:

P(X = k) = C(n, k) · pk · (1 − p)n − k
for k = 0, 1, ..., n. E[X] = np and Var(X) = np(1 − p). A binomial count is just the sum of n independent Bernoulli trials.

Worked example: free-to-paid conversions

200 users sign up, each converting independently with probability 0.08. X ~ Binomial(n = 200, p = 0.08).

Pythonsignups = stats.binom(n=200, p=0.08) print(f"Expected conversions: {signups.mean():.0f}") print(f"SD: {signups.std():.2f}") print(f"P(exactly 16 convert): {signups.pmf(16):.4f}") print(f"P(10 or fewer convert): {signups.cdf(10):.4f}") print(f"P(more than 20 convert): {signups.sf(20):.4f}")
OutputExpected conversions: 16 SD: 3.84 P(exactly 16 convert): 0.1034 P(10 or fewer convert): 0.0691 P(more than 20 convert): 0.1225
0 5 10 15 20 25 30 35 mean = 16 10 or fewer: 6.9% more than 20: 12.2% number of the 200 users who convert 0 5 10 15 20 25 30 35 mean = 16 10 or fewer: 6.9% more than 20: 12.2% number of the 200 users who convert
Binomial(200, 0.08). The most likely single outcome is 16, but it only happens about 10% of the time. A weak cohort of 10 or fewer happens about 1 time in 14, purely by chance.

16 conversions is the most likely single outcome, yet it happens only about 10% of the time. A disappointing cohort of 10 or fewer turns up about 1 time in 14, and a great one of more than 20 about 1 time in 8, with nothing changing except luck. Knowing this range stops a team from over-reacting to one good or bad month.

Another quick example: quality control. A factory's circuit boards have a 3% defect rate, and an inspector checks 50 of them: Binomial(50, 0.03). Expected defects: 1.5. P(no defects at all) ≈ 21.8%. P(more than 3 defects) ≈ 6.3%. If the rule is "more than 3 defects stops the line", the binomial tells the factory it will stop about 1 inspection in 16 even when nothing is wrong. That's the cost of the rule, known in advance.

What "just by chance" looks like in an A/B test

A landing page converts 4.5% of visitors. You show a new version to 1,000 visitors and get 50 conversions instead of the expected 45. Is the new version better? First ask: how often would the old rate produce 50 or more anyway?

Pythonbaseline = stats.binom(n=1000, p=0.045) print(f"Expected conversions: {baseline.mean():.0f}") print(f"P(50 or more, no real change): {baseline.sf(49):.4f}")
OutputExpected conversions: 45 P(50 or more, no real change): 0.2424
25 35 45 55 65 expected 45 50 or more: 24.2% with no real change conversions out of 1,000 visitors at a true rate of 4.5% 25 35 45 55 65 expected 45 50 or more: 24.2% with no real change conversions out of 1,000 visitors at a true rate of 4.5%
If nothing changed, conversions out of 1,000 still scatter widely around 45. The highlighted bars, 50 or more, happen almost 1 time in 4.

Almost 1 in 4. A jump from 45 to 50 is entirely consistent with no real improvement at all. Measuring a result against what chance alone would produce is the core idea of hypothesis testing, which we'll build properly on Day 12 and apply to A/B tests on Day 13. Note that sf(49) gives P(X > 49), which for a count is the same as P(X ≥ 50): Day 8's "watch the endpoints" rule in action.

Real case: Mendel's peas

In 1866 the monk Gregor Mendel published his experiments crossing pea plants. His theory predicted that 3 out of 4 seeds in one generation should be round and 1 in 4 wrinkled. Out of 7,324 seeds, he counted 5,474 round and 1,850 wrinkled, a ratio of 2.96 to 1. Is that close enough to 3 to 1?

If each seed is independently round with probability 0.75, the number of round seeds is Binomial(7,324, 0.75). That gives the yardstick:

Pythonpeas = stats.binom(n=7324, p=0.75) low, high = peas.interval(0.95) print(f"Expected round seeds: {peas.mean():.0f} (SD {peas.std():.1f})") print(f"95% of the time, the count falls between {low:.0f} and {high:.0f}") print(f"Mendel observed 5,474: {(5474 - peas.mean()) / peas.std():.2f} SDs from expected")
OutputExpected round seeds: 5493 (SD 37.1) 95% of the time, the count falls between 5420 and 5565 Mendel observed 5,474: -0.51 SDs from expected

Mendel's count is about half a standard deviation below the prediction, comfortably inside the range chance would produce. The binomial turned "2.96 looks close to 3" into "this is exactly the kind of variation the theory predicts".

When the binomial doesn't apply

Check the four conditions before you reach for it. If you sample without replacement from a small group, such as drawing 10 cards from a deck, the trials aren't independent and p changes after each draw. That calls for a different distribution, the hypergeometric. And if p itself shifts from one batch to another, for example when users convert more readily on some days than others, real counts spread out more than the binomial predicts. With large populations and small samples, the binomial is usually a good approximation anyway.

3. The geometric distribution

How many trials until the first success? The binomial fixes the number of trials and counts successes. The geometric flips the question: keep going until the first success, and count how long it took. How many cold calls until the first meeting is booked? How many attempts until a failed API call goes through?

P(X = k) = (1 − p)k − 1 · p
for k = 1, 2, 3, ...: fail k − 1 times, then succeed on trial k. E[X] = 1/p and Var(X) = (1 − p)/p2.

Quick check: if p = 0.3, the chance the first success comes on trial 3 is 0.7 × 0.7 × 0.3 = 0.147 (fail, fail, succeed).

The mean, 1/p, is about as intuitive as results get. If 20% of calls book a meeting, you expect 1/0.2 = 5 calls per booking. Drop the rate to 5% and it's 20 calls. The rarer the success, the longer the wait.

Two conventions

Some books define the geometric distribution as the number of failures before the first success, starting at 0, with mean (1 − p)/p. SciPy's stats.geom uses the version here, counting trials and starting at 1. Always check which one a formula or library means.

Worked example: the fraud review queue

Each flagged transaction has a 15% chance of being real fraud. X ~ Geometric(0.15) is the number of reviews up to and including the first real case.

Pythonqueue = stats.geom(0.15) print(f"Expected reviews until first fraud: {queue.mean():.1f}") print(f"P(found within 5 reviews): {queue.cdf(5):.4f}") print(f"P(nothing after 10 reviews): {queue.sf(10):.4f}")
OutputExpected reviews until first fraud: 6.7 P(found within 5 reviews): 0.5563 P(nothing after 10 reviews): 0.1969
1 5 10 15 20 25 within 5: 55.6% more than 10: 19.7% reviews until the first confirmed fraud 1 5 10 15 20 25 within 5: 55.6% more than 10: 19.7% reviews until the first confirmed fraud
The tallest bar is always trial 1, whatever p is: each bar is 85% of the one before it. Yet the long tail means about 1 search in 5 takes more than 10 reviews.

The analyst expects to review about 6.7 transactions per confirmed fraud. More than half the time she finds one within 5, but about 1 search in 5 drags past 10 reviews. Two more quick examples:

The memoryless property

The geometric distribution has a surprising property: past failures tell you nothing about the future. After 8 unsuccessful calls, the 9th call still succeeds with probability p, exactly as the first one did. Formally, P(X > m + n | X > m) = P(X > n): having already waited m trials doesn't shorten the remaining wait. Believing you're "due" a success after a run of failures is the gambler's fallacy.

Real case: the night black came up 26 times

On 18 August 1913, at the Monte Carlo Casino, the roulette ball landed on black 26 times in a row. As the streak grew, gamblers bet more and more heavily on red, convinced it was "due". They lost millions of francs. On a single-zero wheel, P(black) = 18/37 on every spin, so any particular run of 26 spins all landing black has a probability of about 1 in 137 million. But after 25 blacks, the chance that the 26th is also black is still just 18/37. The wheel has no memory.

4. The Poisson distribution

Counting events in a fixed window of time or space. How many customers arrive in the next 10 minutes? How many errors occur in an hour? How many typos are on a page? There's no fixed number of trials here, just events happening at some average rate. That's the Poisson distribution, named after the French mathematician Siméon Denis Poisson.

P(X = k) = e−λ · λkk!
for k = 0, 1, 2, ..., where λ (lambda) is the average number of events per window. E[X] = λ and Var(X) = λ: the mean and variance are equal.

Quick check: with λ = 2, P(X = 3) = e−2 × 23 / 3! = (0.1353 × 8)/6 ≈ 0.180.

The Poisson model fits best when three things are roughly true: events happen independently of each other, the average rate stays constant across the window, and two events don't happen at exactly the same instant. Real arrival rates often change with the time of day, so in practice you apply it to windows where the rate is fairly steady, such as "peak hour".

It also scales neatly with time. If events occur at rate λ per hour, the count over t hours is Poisson(λt). 12 support tickets an hour means a 15-minute window follows Poisson(12 × 0.25) = Poisson(3).

Worked example: monitoring production errors

A site logs 4.5 errors per hour during peak traffic. X ~ Poisson(4.5).

Pythonerrors = stats.poisson(4.5) print(f"P(0 errors in an hour): {errors.pmf(0):.4f}") print(f"P(4 or fewer): {errors.cdf(4):.4f}") print(f"P(more than 8): {errors.sf(8):.4f}")
OutputP(0 errors in an hour): 0.0111 P(4 or fewer): 0.5321 P(more than 8): 0.0403
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 more than 8: 4.0% errors in one hour (average 4.5) 0 2 4 6 8 10 12 14 more than 8: 4.0% errors in one hour (average 4.5)
Poisson(4.5). A completely quiet hour is rare (1.1%); more than 8 errors happens in only about 1 hour in 25.

A completely quiet hour is rare, and staying at or below average is about a coin flip. More than 8 errors happens about 1 hour in 25, which gives the on-call team a principled alert threshold: "page someone above 8 errors an hour" will fire about once every 25 peak hours even when nothing is wrong, so it's worth deciding whether that false-alarm rate is acceptable.

Two more quick examples. A coffee shop serves 30 customers an hour on average, so a 10-minute window is Poisson(5): P(more than 8 arrivals) ≈ 6.8%, so one cashier who can handle 8 customers per 10 minutes copes about 93% of the time. And a health department expecting 10 cases of a rare condition in a city, Poisson(10), would see more than 15 cases only about 4.9% of the time, and fewer than 5 only about 2.9% of the time. Both extremes deserve a closer look.

Real case: deaths by horse kick in the Prussian army

In his 1898 book The Law of Small Numbers, the statistician Ladislaus Bortkiewicz analysed a grim dataset: soldiers killed by horse kicks in 10 Prussian army corps over the 20 years from 1875 to 1894, giving 200 "corps-years". Deaths were rare and unpredictable, exactly the setting the Poisson distribution describes. Here's how the real counts compare with a Poisson model:

Pythonimport numpy as np deaths = np.array([0, 1, 2, 3, 4]) # deaths in one corps in one year corps_years = np.array([109, 65, 22, 3, 1]) # how many corps-years had that many all_values = np.repeat(deaths, corps_years) # the 200 individual observations lam = all_values.mean() print(f"Mean deaths per corps-year: {lam:.2f}") print(f"Variance: {all_values.var(ddof=1):.2f}") expected = 200 * stats.poisson.pmf(deaths, lam) for d, obs, exp in zip(deaths, corps_years, expected): print(f"{d} deaths: observed {obs:3d}, Poisson predicts {exp:5.1f}")
OutputMean deaths per corps-year: 0.61 Variance: 0.61 0 deaths: observed 109, Poisson predicts 108.7 1 deaths: observed 65, Poisson predicts 66.3 2 deaths: observed 22, Poisson predicts 20.2 3 deaths: observed 3, Poisson predicts 4.1 4 deaths: observed 1, Poisson predicts 0.6
109 108.7 0 65 66.3 1 22 20.2 2 3 4.1 3 1 0.6 4 deaths from horse kicks in one corps in one year observed (200 corps-years) Poisson(0.61) prediction 109 108.7 0 65 66.3 1 22 20.2 2 3 4.1 3 1 0.6 4 deaths from horse kicks in one corps in one year observed (200 corps-years) Poisson(0.61) prediction
Bortkiewicz's horse-kick data: observed corps-years (purple) against a Poisson model with mean 0.61 (amber). A one-number model reproduces the whole pattern.

A model with a single parameter, the average of 0.61 deaths per corps-year, reproduces the real counts remarkably closely. The data also passes the Poisson's built-in check: its mean (0.61) and variance (0.61) are almost identical.

The mean-equals-variance check

For real count data, compare the sample mean with the sample variance. Roughly equal suggests a Poisson model may fit. If the variance is much larger, which is called overdispersion, counts are burstier than Poisson allows, often because the rate varies (busy days and quiet days, heavy users and light users). A negative binomial model is the usual next step. The same check matters for Poisson regression, the standard regression model for count outcomes such as claims or purchases: if the counts are overdispersed, a negative binomial regression is usually the better choice.

Choosing the right distribution

DistributionQuestion it answersParametersE[X]Var(X)
Bernoulli(p)Did one yes/no event happen?ppp(1 − p)
Binomial(n, p)How many successes in n independent trials?n, pnpnp(1 − p)
Geometric(p)How many trials until the first success?p1/p(1 − p)/p2
Poisson(λ)How many events in a fixed window?λλλ

Four questions to ask yourself: "Did one thing happen?" points to Bernoulli. "A fixed number of tries, count the successes" points to binomial. "How long until the first success?" points to geometric. "No fixed number of tries, events arrive at a rate" points to Poisson.

Bernoulli(p) one yes/no trial Binomial(n, p) count in n trials Geometric(p) wait for the first yes Poisson(λ) count at a rate add up n of them repeat until a 1 n large, p small with λ = np Bernoulli(p) one yes/no trial Binomial(n, p) count in n trials Geometric(p) wait for the first yes Poisson(λ) count at a rate add up n repeat until a 1 n large, p small with λ = np
The four distributions are one family, all built from the same yes/no trial.

The family connections are exact. A binomial is the sum of n independent Bernoulli trials. A geometric counts Bernoulli trials until the first 1. And when n is very large and p very small, with np = λ held fixed, Binomial(n, p) becomes almost identical to Poisson(λ). That's why Poisson describes rare events among many opportunities, like horse kicks among thousands of soldiers:

Pythonn, p = 100_000, 0.00002 # many chances, each very unlikely: np = 2 for k in range(4): print(f"P(X = {k}): binomial {stats.binom.pmf(k, n, p):.5f} Poisson {stats.poisson.pmf(k, n * p):.5f}")
OutputP(X = 0): binomial 0.13533 Poisson 0.13534 P(X = 1): binomial 0.27067 Poisson 0.27067 P(X = 2): binomial 0.27067 Poisson 0.27067 P(X = 3): binomial 0.18045 Poisson 0.18045

The two agree to four decimal places. A common rule of thumb is that the Poisson approximation works well when n is at least 20 and p is at most 0.05, and it gets better as n grows and p shrinks.

Case study: one ops meeting, four distributions

You're the data scientist at DeliverFast, a food delivery startup. The operations lead opens the weekly meeting with four questions.

"What's the risk that any single order is late?" 9% of deliveries run late, so each order is Bernoulli(0.09). E[X] = 0.09, Var(X) ≈ 0.082. Lateness is the exception, not the rule.

"Out of today's 120 deliveries, how many will be late?" Binomial(120, 0.09). Expect 10.8. P(exactly 10 late) ≈ 12.6%. P(more than 15 late) ≈ 7.3%, the threshold ops uses to trigger a staffing review.

"A new restaurant just joined. How soon until its first 5-star review?" 25% of its orders earn 5 stars, so the wait is Geometric(0.25). Expected wait: 1/0.25 = 4 orders. P(a 5-star review within the first 3 orders) ≈ 57.8%.

"How many support tickets in the next 10 minutes of the dinner rush?" Tickets arrive at 6 per hour, so a 10-minute window is Poisson(6 × 10/60) = Poisson(1). P(no tickets) ≈ 36.8%. P(more than 3 tickets) ≈ 1.9%, rare enough that 4 or more tickets in ten minutes is a real signal worth paging a second agent for.

Pythonprint(f"Late risk, one order: {stats.bernoulli(0.09).mean():.2f}") print(f"P(more than 15 of 120 late): {stats.binom(120, 0.09).sf(15):.3f}") print(f"P(5-star within 3 orders): {stats.geom(0.25).cdf(3):.3f}") print(f"P(more than 3 tickets/10 min): {stats.poisson(1).sf(3):.3f}")
OutputLate risk, one order: 0.09 P(more than 15 of 120 late): 0.073 P(5-star within 3 orders): 0.578 P(more than 3 tickets/10 min): 0.019

Four questions, four mechanisms: a single yes-or-no outcome, a count of successes in fixed trials, a wait for the first success, and a count of events in a time window. Recognising which one you're looking at is the real skill. After that, the formula is bookkeeping.

Key takeaways

Quick check

Five questions to test whether this stuck. The answers are one click away.

A website has 95% uptime, checked once a day. Is "the site is up today" Bernoulli, binomial, geometric or Poisson?

Bernoulli: one yes/no check, X ~ Bernoulli(0.95). "How many of the next 30 days will it be up?" would be Binomial(30, 0.95), assuming days are independent.

A support line averages 2 calls per minute. Which distribution describes the number of calls in a 5-minute window, and with what parameter?

Poisson, with λ = 2 × 5 = 10. There's no fixed number of trials, just calls arriving at a steady average rate.

Why is trial 1 always the most likely value of a geometric distribution, however small p is?

P(X = k) = (1 − p)k − 1 p, and (1 − p)k − 1 shrinks every time k grows, because 1 − p is less than 1. Trial 1 needs no failures first, so it always has the highest probability, even when the average wait is long.

A sales rep has made 15 calls without a booking at a 20% booking rate. Is the next call more likely to succeed?

No. If calls are independent, the next one still has a 20% chance, exactly like the first. The geometric distribution is memoryless, and believing otherwise is the gambler's fallacy. (A long run of failures might, however, be a reason to check whether the 20% rate still applies to this list of prospects.)

A count variable has a sample mean of 4.2 and a sample variance of 11.6. Should you use a Poisson model?

Probably not. A Poisson model's variance equals its mean, but here the variance is nearly three times larger. That overdispersion suggests a negative binomial model instead.

Next up, Day 10 Continuous Probability Distributions

Counts aren't the whole story. Next: the uniform, normal and exponential distributions for measurements like time, size and money, and why the normal distribution sits at the heart of statistics.