EDA Case Study: Retail Customer Data

Jr Codex Data Science Notes

Level: Intermediate–Advanced Prerequisites: Chapter 4 Time to complete: ~35 minutes

Table of Contents

1. The Scenario
2. Step 1 — First Contact
3. Step 2 — Cleaning What We Found
4. Step 3 — Univariate Analysis
5. Step 4 — Bivariate Analysis
6. Step 5 — Outlier Investigation
7. Step 6 — Multivariate Check
8. Step 7 — Writing the Findings
9. Where to Go From Here

1. The Scenario

You've just received retail_customers.csv from a mid-size retailer, who wants to understand what distinguishes their high-value customers. Following Chapter 1's EDA sequence exactly, start to finish, on a single realistic dataset:

retail_customers.csv — columns
─────────────────────────────────────────
  customer_id       unique identifier
  age                customer's age
  signup_date         when they joined (string, needs parsing)
  region               "North", "South", "East", "West"
  total_spend           lifetime purchase total ($)
  num_orders             lifetime order count
  last_purchase_days_ago  days since their most recent order
─────────────────────────────────────────

2. Step 1 — First Contact

import pandas as pd
 
df = pd.read_csv("retail_customers.csv")
 
print(df.shape)          # (5000, 7)
print(df.head())
print(df.dtypes)
# customer_id                  int64
# age                        float64
# signup_date                 object   ← should be a date, currently a string
# region                       object
# total_spend                float64
# num_orders                    int64
# last_purchase_days_ago         int64
 
print(df.isna().sum())
# age                  120
# signup_date            0
# region                45
# total_spend            0
# num_orders             0
# last_purchase_days_ago 0
 
print(df.duplicated().sum())      # 8 — a handful of exact duplicate rows

Immediate observations, following Chapter 1's habit of asking questions right away:

• age is 2.4% missing, region is 0.9% missing — small enough that deletion (Module 2, Ch.4) is a reasonable option, but worth checking why first.
• signup_date needs parsing (Module 2, Ch.3) before it's usable.
• 8 duplicate rows need investigation before any statistic is trusted.

3. Step 2 — Cleaning What We Found

Applying Module 2's techniques directly:

# Remove exact duplicates (Module 2, Ch.4)
df = df.drop_duplicates()
 
# Parse the date column (Module 2, Ch.3)
df["signup_date"] = pd.to_datetime(df["signup_date"], errors="coerce")
 
# Standardize region text, in case of inconsistent casing (Module 2, Ch.3)
df["region"] = df["region"].str.strip().str.title()
 
# Check whether age/region missingness looks MAR (Module 2, Ch.4)
print(df.groupby(df["region"].isna())["total_spend"].mean())
# If missing-region customers have notably different spend, that's a MAR signal
# worth flagging rather than blindly dropping
 
# Decision: age and region missingness is small (<3%) and shows no strong
# relationship to total_spend — safe to drop these rows for this analysis
df_clean = df.dropna(subset=["age", "region"])
print(f"Rows before cleaning: {len(df)}, after: {len(df_clean)}")

4. Step 3 — Univariate Analysis

import matplotlib.pyplot as plt
 
print(df_clean["total_spend"].describe())
print(f"Skewness: {df_clean['total_spend'].skew():.2f}")      # likely notably positive
 
fig, axes = plt.subplots(1, 2, figsize=(12, 4))
axes[0].hist(df_clean["total_spend"], bins=40)
axes[0].set_title("Total Spend Distribution")
axes[1].boxplot(df_clean["total_spend"])
axes[1].set_title("Total Spend Box Plot")

Finding: total_spend is right-skewed (Module 1, Ch.1/Ch.3) — most customers spend modestly, with a long tail of high spenders. This immediately tells us: report the median, not the mean, when describing a "typical" customer, and expect the outlier investigation (Step 5) to matter here.

print(df_clean["region"].value_counts(normalize=True) * 100)
# North    32.1%
# South    28.4%
# East     22.0%
# West     17.5%   ← notably smaller customer base

5. Step 4 — Bivariate Analysis

import seaborn as sns
import matplotlib.pyplot as plt
 
# Numeric vs numeric (Chapter 3, Section 2)
print(df_clean[["age", "num_orders", "total_spend"]].corr())
#                  age  num_orders  total_spend
# age             1.00        0.12         0.18
# num_orders      0.12        1.00         0.81   ← strong! more orders → more spend, unsurprising but worth confirming
# total_spend     0.18        0.81         1.00
 
plt.scatter(df_clean["num_orders"], df_clean["total_spend"], alpha=0.3)
plt.xlabel("Number of Orders")
plt.ylabel("Total Spend")
 
# Categorical vs numeric (Chapter 3, Section 3)
print(df_clean.groupby("region")["total_spend"].median().sort_values(ascending=False))
# West     412.50   ← smallest customer base, but HIGHEST median spend — notable!
# North    285.00
# South    260.75
# East     245.00

Finding: the West region has the fewest customers but the highest median spend per customer — a pattern invisible from the univariate region counts alone. This is exactly the kind of question that should trigger a follow-up (is West a newer, premium market? A different customer acquisition channel?) — EDA surfaces the question; answering it needs either domain knowledge or a deeper multivariate look (Step 6).

6. Step 5 — Outlier Investigation

q1 = df_clean["total_spend"].quantile(0.25)
q3 = df_clean["total_spend"].quantile(0.75)
iqr = q3 - q1
upper_bound = q3 + 1.5 * iqr
 
outliers = df_clean[df_clean["total_spend"] > upper_bound]
print(f"{len(outliers)} customers flagged as high-spend outliers")
print(outliers[["customer_id", "total_spend", "num_orders", "region"]].sort_values(
    "total_spend", ascending=False
).head(10))

Applying Chapter 4's decision framework:

• These aren't impossible values (no negative spend, no absurd magnitudes) — not data errors.
• They correspond to customers with proportionally high num_orders too (consistent with the strong correlation found in Step 4) — internally consistent, not a glitch.
• Decision: keep these rows. They're very likely genuine high-value customers — exactly the population this entire analysis is meant to understand. Removing them would defeat the purpose of the investigation.

# Rather than remove them, TAG them — directly useful for the analysis's actual goal
df_clean["is_high_value"] = df_clean["total_spend"] > upper_bound
print(df_clean.groupby("is_high_value")[["age", "num_orders"]].mean())

7. Step 6 — Multivariate Check

Following up on the West-region finding from Step 4, and Chapter 3's Simpson's Paradox warning — does the "West has higher spend" pattern hold up once num_orders (a known strong driver of spend) is accounted for?

import seaborn as sns
 
sns.scatterplot(data=df_clean, x="num_orders", y="total_spend", hue="region", alpha=0.5)

# Does West still lead in spend PER ORDER, not just total? A fairer comparison.
df_clean["spend_per_order"] = df_clean["total_spend"] / df_clean["num_orders"]
print(df_clean.groupby("region")["spend_per_order"].median().sort_values(ascending=False))
# West     38.20   ← STILL highest — the pattern holds up, not just an artifact of order count
# North    24.10
# South    22.80
# East     21.50

Finding confirmed: West customers spend more per order, not just more in total — ruling out "they just order more often" as the explanation, and making this a genuinely interesting business question rather than a statistical artifact.

8. Step 7 — Writing the Findings

Following Chapter 1's documentation habit — this is the actual deliverable of an EDA session:

EDA Findings: retail_customers.csv
─────────────────────────────────────────
Dataset: 5,000 rows → 4,847 after removing 8 duplicates and 145 rows
          with missing age/region (deemed safe to drop; no strong
          relationship found between missingness and total_spend)

Key Findings:
 1. total_spend is right-skewed — report MEDIAN ($280), not mean, as "typical"
 2. num_orders correlates strongly with total_spend (r=0.81) — expected,
    but now confirmed rather than assumed
 3. West region has the smallest customer base (17.5%) but the HIGHEST
    median spend per order ($38.20 vs $21.50-$24.10 elsewhere) — held up
    even after controlling for order count. WORTH INVESTIGATING: possible
    different acquisition channel or product mix in West.
 4. 187 customers flagged as high-value outliers (spend > $650) — kept
    in the dataset; they appear genuine (consistent order counts) and are
    the exact population the business wants to understand better.

Open Questions for Follow-Up:
 - What's driving West's higher per-order spend? (needs data outside
   this dataset — marketing channel, product category mix)
 - Are the 187 high-value customers a candidate segment for the
   Machine Learning notes' classification techniques (predicting
   "will become high-value")?
─────────────────────────────────────────

9. Where to Go From Here

This case study followed the exact sequence from Chapter 1 — first contact, cleaning, univariate, bivariate, outliers, multivariate, and a written summary — on one realistic dataset. Every technique traced back to a specific earlier chapter, which is the point: EDA isn't a separate skill from statistics and wrangling, it's where those skills get applied together with judgment.

Natural next steps for a dataset like this one:

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index