Probability Distributions

Jr Codex Data Science Notes

Level: Beginner–Intermediate Prerequisites: Chapter 2 Time to complete: ~25 minutes

Table of Contents

1. What is a Probability Distribution?
2. Discrete vs Continuous Distributions
3. The Uniform Distribution
4. The Binomial Distribution
5. The Poisson Distribution
6. The Normal (Gaussian) Distribution
7. The Empirical Rule
8. Summary & Next Steps

1. What is a Probability Distribution?

A probability distribution describes how probabilities are spread across all possible values of a random variable — essentially, a "shape" that tells you which values are likely and which are rare.

A Distribution Answers:
─────────────────────────────────────────
  "If I pick a random value from this process, what's
   the CHANCE it falls near X, versus near Y?"
─────────────────────────────────────────

Different real-world processes naturally produce different shapes of distribution — recognizing which shape you're dealing with tells you which statistical tools apply.

2. Discrete vs Continuous Distributions

# Discrete: outcomes are distinct, countable values (e.g. number of heads in 10 flips)
# Continuous: outcomes can be ANY value in a range (e.g. height, temperature, time)

3. The Uniform Distribution

Every outcome is equally likely — the simplest distribution shape.

import numpy as np
 
# Discrete uniform: rolling a fair die
die_roll = np.random.randint(1, 7, size=10000)      # 10,000 rolls
import matplotlib.pyplot as plt
plt.hist(die_roll, bins=6)      # flat, roughly EQUAL height bars — every outcome equally likely
 
# Continuous uniform: any decimal value between 0 and 1, equally likely
uniform_samples = np.random.uniform(0, 1, size=10000)

Uniform Distribution Shape
─────────────────────────────────────────
  ████ ████ ████ ████ ████ ████
   1    2    3    4    5    6      ← all bars roughly EQUAL height
─────────────────────────────────────────

4. The Binomial Distribution

Models the number of successes in a fixed number of independent yes/no trials (each with the same success probability) — the natural extension of the coin-flip idea from Chapter 2.

from scipy.stats import binom
import numpy as np
 
# 10 coin flips, probability of heads = 0.5 — how many heads would we typically see?
n_trials = 10
p_success = 0.5
 
# P(exactly 5 heads out of 10 flips)
p_exactly_5 = binom.pmf(5, n_trials, p_success)
print(p_exactly_5)      # ~0.246
 
# Simulate it directly to build intuition
simulated_flips = np.random.binomial(n_trials, p_success, size=10000)
print(np.mean(simulated_flips == 5))      # ~0.246 — matches the formula

Binomial — the Formula's Ingredients
─────────────────────────────────────────
  n = number of trials           (e.g. 10 flips)
  p = probability of SUCCESS      (e.g. 0.5 for a fair coin)
  k = number of successes we're asking about   (e.g. exactly 5 heads)
─────────────────────────────────────────

Real-world use: conversion rate experiments (Module 5's A/B testing) — "if the true conversion rate is 5%, how many conversions would we expect out of 1,000 visitors?" is a direct binomial question.

# Expected conversions out of 1000 visitors, at a 5% true conversion rate
expected_conversions = 1000 * 0.05
print(expected_conversions)      # 50.0 — the MEAN of a binomial distribution is n * p

5. The Poisson Distribution

Models the number of times an event occurs in a fixed interval (of time or space), when events happen independently at some average rate.

from scipy.stats import poisson
import numpy as np
 
# A call center averages 4 calls per minute. What's P(exactly 6 calls in a given minute)?
average_rate = 4
p_exactly_6 = poisson.pmf(6, average_rate)
print(p_exactly_6)      # ~0.104
 
simulated_minutes = np.random.poisson(average_rate, size=10000)
print(np.mean(simulated_minutes == 6))      # ~0.104 — matches

Binomial vs Poisson — When to Use Which
─────────────────────────────────────────
  Binomial: FIXED number of trials, each with a known success probability
            (e.g. "out of 1000 visitors, how many convert?")

  Poisson:  events happening at some average RATE, over continuous time/space
            (e.g. "how many customer support tickets arrive per hour?")
─────────────────────────────────────────

Real-world use: website traffic spikes, server error rates, customer arrivals — anywhere you're counting "how many events in this window," not "how many successes out of N tries."

6. The Normal (Gaussian) Distribution

The single most important continuous distribution in statistics — the classic "bell curve." Many natural phenomena (heights, measurement errors, test scores) approximate it, and — as Chapter 4 will show — sample means tend toward it regardless of the underlying data's own shape.

import numpy as np
from scipy.stats import norm
 
# Heights of adult men, approximately: mean=175cm, std dev=7cm
heights = np.random.normal(loc=175, scale=7, size=10000)
 
print(np.mean(heights))       # ~175 — close to the specified mean
print(np.std(heights))          # ~7 — close to the specified std dev
 
# P(a random person is taller than 189cm)? — using the CDF (cumulative distribution function)
p_taller_than_189 = 1 - norm.cdf(189, loc=175, scale=7)
print(p_taller_than_189)      # ~0.023 — about 2.3%

The Normal Distribution's Shape
─────────────────────────────────────────
              ___
           _-'   '-_
         /           \
       _/             \_
    __/                 \__
  (symmetric, bell-shaped, centered on the mean)
   mean - 3σ   mean    mean + 3σ
─────────────────────────────────────────

A normal distribution is fully described by just two numbers: its mean (center) and standard deviation (spread) — this is why those two descriptive statistics (Chapter 1) are so central to so much of statistics.

7. The Empirical Rule

For any normal distribution, a fixed percentage of data falls within 1, 2, and 3 standard deviations of the mean — worth memorizing, since it's used constantly for quick sanity-checks.

The Empirical Rule ("68-95-99.7 Rule")
─────────────────────────────────────────
  68% of data falls within  ±1 standard deviation of the mean
  95% of data falls within  ±2 standard deviations of the mean
  99.7% of data falls within ±3 standard deviations of the mean
─────────────────────────────────────────

import numpy as np
 
heights = np.random.normal(loc=175, scale=7, size=100000)
 
mean = np.mean(heights)
std = np.std(heights)
 
within_1_std = np.mean((heights > mean - std) & (heights < mean + std))
within_2_std = np.mean((heights > mean - 2*std) & (heights < mean + 2*std))
 
print(f"{within_1_std:.1%}")      # ~68.0% — matches the empirical rule
print(f"{within_2_std:.1%}")        # ~95.0% — matches

This is exactly the reasoning behind flagging something as "unusual" — a value more than 2-3 standard deviations from the mean is, by definition, rare under a normal distribution, and is one of the standard statistical approaches to outlier detection (revisited in Module 3).

8. Summary & Next Steps

Key Takeaways

• A distribution describes how probability is spread across a random variable's possible values — recognizing the shape tells you which tools apply.
• Binomial models successes out of a fixed number of trials; Poisson models event counts over a fixed interval at some average rate; Uniform means every outcome is equally likely.
• The Normal distribution is fully described by its mean and standard deviation, and many natural processes (plus, critically, sample means — Chapter 4) approximate it.
• The empirical rule (68-95-99.7) gives a fast way to judge how unusual a value is, purely from how many standard deviations it sits from the mean.

Concept Check

1. You're modeling "number of support tickets per hour" — is that binomial or Poisson, and why?
2. What two numbers fully describe a normal distribution?
3. Under the empirical rule, roughly what percentage of data falls within 2 standard deviations of the mean?

Next Chapter

→ Chapter 4: Sampling & the Central Limit Theorem

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index