Data Preparation For ML
Handling Imbalanced Datasets
n_fraud = 50 # only 0.5% of transactions are fraudulent
Jr Codex ML Notes
Level: Intermediate–Advanced Prerequisites: Chapter 4 Time to complete: ~25 minutes
Table of Contents
- What is Class Imbalance?
- Why Accuracy Lies on Imbalanced Data
- Oversampling the Minority Class
- SMOTE — Synthetic Minority Oversampling
- Undersampling the Majority Class
- Class Weights — Fixing the Algorithm Instead of the Data
- Choosing the Right Evaluation Metric
- A Complete Workflow
- Summary & Next Steps
1. What is Class Imbalance?
Class imbalance occurs when a classification problem's classes are far from equally represented — one class dominates the dataset, while the other (often the one you actually care most about) is rare.
import numpy as np
# A realistic fraud detection dataset
n_total = 10000
n_fraud = 50 # only 0.5% of transactions are fraudulent
y = np.array([0] * (n_total - n_fraud) + [1] * n_fraud)
print(f"Fraud rate: {y.mean():.3%}") # 0.500%Common Real-World Imbalanced Problems
─────────────────────────────────────────
Fraud detection: fraud is rare relative to legitimate
transactions
Disease diagnosis: most patients screened DON'T have
the rare condition being tested for
Manufacturing defects: most produced items are NOT defective
Churn prediction: most customers, in most periods,
do NOT churn
─────────────────────────────────────────
This connects directly to Module 4, Chapter 1's classification metrics discussion — imbalance is precisely why "accuracy" alone is such a dangerously misleading metric for these problems, as Section 2 demonstrates concretely.
2. Why Accuracy Lies on Imbalanced Data
# A "model" that NEVER predicts fraud, no matter what — a completely useless model
def lazy_model_predict(X):
return np.zeros(len(X)) # always predicts "not fraud"
predictions = lazy_model_predict(y)
accuracy = (predictions == y).mean()
print(f"Accuracy of a model that NEVER predicts fraud: {accuracy:.3%}") # 99.500%!The Core Problem
─────────────────────────────────────────
A model that is COMPLETELY USELESS (never catches a single
case of fraud) still achieves 99.5% "accuracy" — simply
because fraud is so rare that predicting "never fraud" is
right almost all the time.
This is EXACTLY why Module 4's classification metrics chapter
insists on precision, recall, and F1-score for imbalanced
problems — accuracy alone is essentially meaningless here.
─────────────────────────────────────────
3. Oversampling the Minority Class
The simplest fix at the data level: duplicate examples from the minority class so the training set is more balanced.
import pandas as pd
from sklearn.utils import resample
df = pd.DataFrame({
"feature1": np.random.rand(1000),
"is_fraud": [0]*990 + [1]*10,
})
majority = df[df["is_fraud"] == 0]
minority = df[df["is_fraud"] == 1]
minority_oversampled = resample(
minority, replace=True, n_samples=len(majority), random_state=42
)
balanced_df = pd.concat([majority, minority_oversampled])
print(balanced_df["is_fraud"].value_counts()) # now roughly 990/990The Risk of Naive Oversampling
─────────────────────────────────────────
Simply DUPLICATING minority examples means the model sees the
EXACT SAME few fraud cases repeated many times — it can
memorize those SPECIFIC examples (Module 1, Ch.4's overfitting
concern) rather than learning the GENERAL pattern of what
fraud looks like.
─────────────────────────────────────────
4. SMOTE — Synthetic Minority Oversampling
SMOTE (Synthetic Minority Oversampling Technique) fixes naive oversampling's weakness by generating new, synthetic minority examples, rather than duplicating existing ones exactly.
from imblearn.over_sampling import SMOTE
X = df[["feature1"]].values
y = df["is_fraud"].values
smote = SMOTE(random_state=42)
X_resampled, y_resampled = smote.fit_resample(X, y)
print(f"Original: {len(y)}, After SMOTE: {len(y_resampled)}")
print(pd.Series(y_resampled).value_counts())How SMOTE Creates Synthetic Examples
─────────────────────────────────────────
For each minority example, SMOTE:
1. Finds its K nearest minority-class NEIGHBORS
(directly reusing Module 4's KNN distance concept)
2. Picks a random point ALONG THE LINE connecting the
original example and one of its neighbors
3. This new point becomes a SYNTHETIC minority example —
plausible, but not an exact duplicate of any real one
─────────────────────────────────────────
Naive Oversampling vs SMOTE
─────────────────────────────────────────
Naive: ● ●●●●● ● (exact duplicates of the SAME few points)
SMOTE: ● ● ○ ● ○ ● ○ ● (○ = NEW synthetic points,
interpolated between real ones —
more DIVERSE, less prone to
memorization)
─────────────────────────────────────────
Important caveat: SMOTE must be applied only to the training set, after the train/test split (Module 1, Chapter 5) — applying it before splitting would leak synthetic-but-derived-from-test-data information into evaluation, another form of data leakage.
5. Undersampling the Majority Class
The opposite approach: remove examples from the majority class instead of adding to the minority class.
majority_undersampled = resample(
majority, replace=False, n_samples=len(minority), random_state=42
)
balanced_df_under = pd.concat([majority_undersampled, minority])
print(balanced_df_under["is_fraud"].value_counts()) # now 10/10Oversampling vs Undersampling — the Trade-off
─────────────────────────────────────────
Oversampling (Sections 3-4): keeps ALL available data —
good when data is scarce
overall
Undersampling (this section): DISCARDS potentially useful
majority-class data — risky
when the majority class
already has LIMITED
examples, but can be
practical when the majority
class is so large that
training time/memory is a
genuine constraint
─────────────────────────────────────────
6. Class Weights — Fixing the Algorithm Instead of the Data
Rather than modifying the data at all, many algorithms let you directly tell them to penalize mistakes on the minority class more heavily during training.
from sklearn.linear_model import LogisticRegression
# class_weight='balanced' automatically weights classes INVERSELY
# proportional to their frequency — rare classes get HIGHER weight
model = LogisticRegression(class_weight="balanced")
model.fit(X, y)
# Equivalent to MANUALLY specifying weights:
model_manual = LogisticRegression(class_weight={0: 1, 1: 99}) # minority class weighted 99x moreWhy This Can Be Preferable to Resampling
─────────────────────────────────────────
- No DUPLICATED or SYNTHETIC data — trains on the ORIGINAL,
real data throughout
- Often SIMPLER to implement (usually just one parameter)
- Most major algorithms across Modules 3-5 (logistic
regression, SVM, decision trees, random forests) support a
`class_weight` parameter directly
─────────────────────────────────────────
7. Choosing the Right Evaluation Metric
Directly connecting to Module 4's classification metrics chapter — the evaluation metric matters as much as the resampling technique itself, echoing Section 2's warning.
Metrics That Remain Meaningful Under Imbalance (Full Depth: Module 4, Ch.6)
─────────────────────────────────────────
Precision: of everything flagged as fraud, how much
ACTUALLY was fraud?
Recall: of all ACTUAL fraud, how much did we catch?
F1-score: the harmonic mean balancing precision
and recall
ROC-AUC / PR-AUC: threshold-independent measures of
overall discriminative ability
(PR-AUC specifically preferred for
SEVERE imbalance)
─────────────────────────────────────────
Never rely on accuracy alone for an imbalanced problem — Section 2's "99.5% accurate, completely useless" model is the permanent cautionary example to keep in mind whenever imbalance is present.
8. A Complete Workflow
Combining every technique in this chapter into one correct, leak-free pipeline:
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from imblearn.over_sampling import SMOTE
# 1. Split FIRST (Module 1, Ch.5)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, stratify=y, random_state=42 # stratify preserves imbalance ratio in the split
)
# 2. Apply SMOTE ONLY to training data
smote = SMOTE(random_state=42)
X_train_resampled, y_train_resampled = smote.fit_resample(X_train, y_train)
# 3. Train on the RESAMPLED training data
model = LogisticRegression()
model.fit(X_train_resampled, y_train_resampled)
# 4. Evaluate on the ORIGINAL, UNTOUCHED (imbalanced) test set —
# this reflects REAL-WORLD imbalance, which resampling training
# data does NOT change
predictions = model.predict(X_test)
print(classification_report(y_test, predictions))Why the TEST Set Stays Imbalanced
─────────────────────────────────────────
Resampling is a TRAINING-time technique to help the model
LEARN the minority class's pattern better — it should NEVER
be applied to the test set, since the test set's job (Module
1, Ch.5) is to honestly reflect REAL-WORLD performance, and
the real world is genuinely imbalanced.
─────────────────────────────────────────
9. Summary & Next Steps
Key Takeaways
- Class imbalance means one class dominates a classification dataset — common in fraud detection, disease diagnosis, and churn prediction, where the rare class is often the one that matters most.
- Accuracy is dangerously misleading under imbalance — a model that never predicts the minority class can still achieve near-perfect accuracy while being completely useless.
- SMOTE generates synthetic (not duplicated) minority examples via interpolation between real neighbors, reducing the overfitting risk of naive oversampling.
- Class weights fix the algorithm's training process directly, without modifying the data at all — often simpler than resampling.
- Resampling techniques apply only to training data; the test set must remain in its original, real-world imbalanced state to give an honest performance estimate.
Module 2 Complete — Next Module
You can now take raw data and prepare it fully for any algorithm in this curriculum: encoding categorical variables, scaling features, engineering and selecting the most useful ones, and handling class imbalance correctly. Module 3 begins the algorithms themselves, starting with regression — the first branch of supervised learning from Module 1's paradigm map.
Concept Check
- Why is a 99.5% accurate fraud model potentially worthless, and what metrics would reveal the problem?
- How does SMOTE differ from naive duplication-based oversampling?
- Why must SMOTE (or any resampling) be applied only to the training set, never the test set?
Next Chapter
→ Module 3: Supervised Learning — Regression
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index