Descriptive Statistics

Jr Codex Data Science Notes

Level: Beginner Prerequisites: Python: NumPy & Pandas Primers Time to complete: ~25 minutes

Table of Contents

1. What is Descriptive Statistics?
2. Measures of Central Tendency
3. Measures of Spread
4. Quartiles & the Five-Number Summary
5. Skewness — When the Data Isn't Symmetric
6. Doing All of This with Pandas
7. Summary & Next Steps

1. What is Descriptive Statistics?

Descriptive statistics summarizes a dataset with a small number of numbers — instead of staring at 10,000 raw rows, you look at a handful of values that describe the whole shape of the data.

Raw Data (10,000 rows)                  Descriptive Summary
─────────────────────────────         ─────────────────────────────
  [23, 45, 12, 67, 34, ...]              mean = 41.2
  (impossible to "see" the pattern)       median = 38.0
                                            std dev = 12.4
                                             (the shape, at a glance)
─────────────────────────────         ─────────────────────────────

Every chapter in this module builds toward being able to look at a dataset and immediately know what questions to ask — descriptive statistics is the first, most basic pass.

2. Measures of Central Tendency

"Where is the center of the data?" — three different, complementary answers:

import numpy as np
 
salaries = np.array([45000, 48000, 50000, 52000, 55000, 250000])
 
print(np.mean(salaries))       # 83333.33 — average: sum divided by count
print(np.median(salaries))       # 51000.0 — the MIDDLE value when sorted

from scipy import stats
 
scores = np.array([85, 90, 85, 78, 85, 92])
print(stats.mode(scores, keepdims=True))   # mode=85 — the MOST FREQUENT value

The salary example above is the classic illustration of why this matters: the mean (83,333) is pulled way up by one outlier (250,000), while the median (51,000) reflects what a "typical" salary in that group actually looks like. When data might contain outliers, prefer the median for a "typical value" claim.

3. Measures of Spread

Central tendency alone can be misleading — two datasets can share the same mean but look completely different:

class_a = np.array([70, 72, 68, 71, 69])       # tightly clustered
class_b = np.array([40, 90, 50, 95, 45])         # wildly spread out
 
print(np.mean(class_a))      # 70.0
print(np.mean(class_b))        # 64.0 — close, but the DISTRIBUTION is totally different

print(np.var(class_a))       # variance — average squared distance from the mean
print(np.std(class_a))         # standard deviation — variance's square root, same UNITS as the data
 
print(np.std(class_a))          # ~1.26 — tightly clustered
print(np.std(class_b))            # ~23.85 — much more spread out

Variance & Standard Deviation
─────────────────────────────────────────
  variance   = average of (each value - mean)²
  std dev     = √variance
             → same units as the original data (unlike variance, which is SQUARED units)
─────────────────────────────────────────

Rule of thumb: always report a measure of spread (std dev) alongside a measure of center (mean) — "average salary is $70k" means very different things depending on whether the std dev is $5k or $50k.

The range is the simplest (and crudest) spread measure:

print(np.max(class_b) - np.min(class_b))     # 55 — just the max minus the min

4. Quartiles & the Five-Number Summary

Quartiles divide sorted data into four equal parts — a more robust alternative to mean/std dev when outliers are a concern.

data = np.array([12, 15, 18, 22, 25, 28, 30, 35, 42, 50, 95])
 
q1 = np.percentile(data, 25)      # 18.0 — 25% of data falls below this
q2 = np.percentile(data, 50)        # 28.0 — the median
q3 = np.percentile(data, 75)          # 42.0 — 75% of data falls below this
 
iqr = q3 - q1                           # Interquartile Range — spread of the MIDDLE 50%
print(f"IQR: {iqr}")                      # 24.0

The Five-Number Summary
─────────────────────────────────────────
  Min ─────── Q1 ─────── Median (Q2) ─────── Q3 ─────── Max
   12          18              28              42          95
       (this is exactly what a box plot, Module 4, visualizes)
─────────────────────────────────────────

The IQR is also the standard tool for a simple outlier rule, covered fully in Module 3's Outlier Detection chapter:

lower_bound = q1 - 1.5 * iqr
upper_bound = q3 + 1.5 * iqr
print(f"Outlier bounds: [{lower_bound}, {upper_bound}]")   # [-9.0, 69.0]
print(data[(data < lower_bound) | (data > upper_bound)])     # [95] — flagged as an outlier

5. Skewness — When the Data Isn't Symmetric

Skewness measures asymmetry — whether one "tail" of the distribution is longer than the other.

from scipy.stats import skew
 
symmetric_data = np.array([10, 20, 30, 40, 50])
right_skewed = np.array([10, 12, 13, 15, 90])       # a few very high values pull the tail right
 
print(skew(symmetric_data))      # ~0 — roughly symmetric
print(skew(right_skewed))          # positive — right-skewed (long tail to the right)

Skewness, Visualized
─────────────────────────────────────────
  Left-skewed (negative)     Symmetric (≈0)      Right-skewed (positive)
       __                        __                    __
      /  \___                  _/  \_                _/  \___
   __/       \              __/      \__          __/        \_____
  (long tail LEFT)         (mean ≈ median)       (long tail RIGHT — e.g. income, salaries)
─────────────────────────────────────────

Practical implication: the salary example from Section 2 is right-skewed — this is exactly why median beats mean there, and it's the same reason income and house-price data are almost always summarized by median in real reporting.

6. Doing All of This with Pandas

In practice, you rarely compute these one at a time — pandas' .describe() (introduced in the Python notes' Pandas Primer) gives you the whole picture in one call:

import pandas as pd
 
df = pd.DataFrame({
    "salary": [45000, 48000, 50000, 52000, 55000, 250000],
    "age": [25, 28, 30, 35, 40, 50],
})
 
print(df.describe())
#              salary        age
# count       6.000000    6.000000
# mean    83333.333333   34.666667
# std     81234.567890    8.960745
# min     45000.000000   25.000000
# 25%     48500.000000   28.750000
# 50%     51000.000000   32.500000
# 75%     54250.000000   38.750000
# max    250000.000000   50.000000
 
print(df["salary"].mean())      # 83333.33
print(df["salary"].median())      # 51000.0
print(df["salary"].std())           # 81234.57
print(df["salary"].skew())            # positive — confirms right skew, matches Section 5

7. Summary & Next Steps

Key Takeaways

• Mean, median, and mode all answer "where's the center?" — but mean is sensitive to outliers while median is robust; check which one is appropriate before reporting a "typical value."
• Variance and standard deviation quantify spread; always pair a spread measure with a center measure when summarizing data.
• Quartiles and the IQR give a robust, outlier-resistant five-number summary — the basis for both the standard outlier rule and box plots (Module 4).
• Skewness tells you whether a distribution is symmetric or has a long tail — right-skewed data (like income) is exactly when median beats mean.

Concept Check

1. Why does the salary example's mean (83,333) differ so much from its median (51,000)?
2. What's the relationship between variance and standard deviation, and why is std dev usually reported instead of variance?
3. What does a positive skew value tell you about a distribution's shape?

Next Chapter

→ Chapter 2: Probability Fundamentals

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index