Dl Engineering And Capstone
Capstone: Building an Image Classifier End-to-End
A conservation organization wants to automatically classify camera-trap images into wildlife species (10 target species common to their region, plus an "empty/n
JrCodex·6 min read
Jr Codex Deep Learning Notes
Level: Advanced Prerequisites: All of Modules 1–9 (Chapters 1-3) Time to complete: ~50 minutes
Table of Contents
- The Scenario
- Phase 1 — Problem Definition
- Phase 2 — Data Preparation
- Phase 3 — Baseline With Transfer Learning
- Phase 4 — Fine-Tuning
- Phase 5 — Rigorous Evaluation
- Phase 6 — Debugging Along the Way
- Phase 7 — Preparing for Deployment
- Where to Go From Here
1. The Scenario
A conservation organization wants to automatically classify camera-trap images into wildlife species (10 target species common to their region, plus an "empty/no animal" category) — a genuine, widely-used real-world computer vision application. This capstone pulls in techniques from nearly every module in this curriculum.
image_data/ — directory structure
─────────────────────────────────────────
train/
deer/ (labeled images)
fox/
bear/
...
empty/ (no animal present — a common camera-trap category)
val/
(same structure)
─────────────────────────────────────────
2. Phase 1 — Problem Definition
Applying the ML Notes' workflow discipline, extended to this curriculum's vision-specific concerns:
Precise Problem Framing
─────────────────────────────────────────
Task type: MULTI-CLASS image classification (Module 4)
— 11 classes (10 species + empty)
Performance measure: per-class RECALL matters most for
RARE species — missing a rare
animal's presence is more costly
than a false positive on a common
species (echoing the ML Notes'
precision/recall tradeoff)
Class imbalance "empty" images vastly OUTNUMBER
concern: any single species in typical
camera-trap data — directly
connects to the ML Notes'
imbalanced data chapter
─────────────────────────────────────────
3. Phase 2 — Data Preparation
Applying Module 4, Chapter 5's augmentation and Chapter 1's custom dataset patterns:
import torch
from torchvision import transforms, datasets
from torch.utils.data import DataLoader, WeightedRandomSampler
# Augmentation appropriate for OUTDOOR wildlife photos (Module 4, Ch.5, Section 3's
# domain-appropriateness point — NO vertical flips needed here, but rotation/color
# jitter make sense for varying camera angles and lighting)
train_transform = transforms.Compose([
transforms.RandomResizedCrop(224, scale=(0.8, 1.0)),
transforms.RandomHorizontalFlip(p=0.5),
transforms.ColorJitter(brightness=0.3, contrast=0.3), # camera-trap lighting varies A LOT
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), # matching ImageNet
# pretrained stats
# (Module 7, Ch.3, Sec.5)
])
val_transform = transforms.Compose([
transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
train_dataset = datasets.ImageFolder("image_data/train", transform=train_transform)
val_dataset = datasets.ImageFolder("image_data/val", transform=val_transform)
# Addressing class imbalance (ML Notes) via a WEIGHTED sampler — oversample RARE species
class_counts = torch.bincount(torch.tensor(train_dataset.targets))
class_weights = 1.0 / class_counts.float()
sample_weights = class_weights[train_dataset.targets]
sampler = WeightedRandomSampler(sample_weights, num_samples=len(sample_weights), replacement=True)
train_loader = DataLoader(train_dataset, batch_size=32, sampler=sampler, num_workers=4) # Ch.1, Sec.3
val_loader = DataLoader(val_dataset, batch_size=32, shuffle=False, num_workers=4)4. Phase 3 — Baseline With Transfer Learning
Applying Module 7's transfer learning — starting with feature extraction, the safest, fastest baseline:
import torch.nn as nn
from torchvision import models
model = models.resnet50(weights="IMAGENET1K_V2") # Module 7, Ch.3
for param in model.parameters(): # feature extraction FIRST (Module 7, Ch.2, Sec.2)
param.requires_grad = False
num_classes = 11 # 10 species + empty
model.fc = nn.Linear(model.fc.in_features, num_classes)
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
# SANITY CHECK before any real training (Chapter 3, Section 2)
sample_images, sample_labels = next(iter(train_loader))
sanity_check_overfit_tiny_batch(
model, nn.CrossEntropyLoss(),
torch.optim.Adam(model.fc.parameters(), lr=0.01),
sample_images[:8].to(device), sample_labels[:8].to(device),
)5. Phase 4 — Fine-Tuning
Applying Module 7, Chapter 2's fine-tuning strategy, after the feature-extraction baseline establishes a working pipeline:
import torch.optim as optim
# Train the new layer first (feature extraction)
optimizer = optim.Adam(model.fc.parameters(), lr=0.001)
criterion = nn.CrossEntropyLoss()
for epoch in range(5):
train_loss = train_one_epoch(model, train_loader, optimizer, criterion, device) # Ch.1
val_loss, val_acc = validate(model, val_loader, criterion, device)
print(f"[Feature extraction] Epoch {epoch+1}: val_acc={val_acc:.4f}")
# THEN unfreeze the last block and fine-tune at a LOW rate (Module 7, Ch.2, Sec.5)
for param in model.layer4.parameters():
param.requires_grad = True
fine_tune_optimizer = optim.AdamW([
{"params": model.layer4.parameters(), "lr": 1e-5},
{"params": model.fc.parameters(), "lr": 1e-4},
], weight_decay=1e-4) # Module 3, Ch.4
scheduler = optim.lr_scheduler.CosineAnnealingLR(fine_tune_optimizer, T_max=10) # Module 3, Ch.5
best_val_acc = 0
for epoch in range(10):
train_loss = train_one_epoch(model, train_loader, fine_tune_optimizer, criterion, device)
val_loss, val_acc = validate(model, val_loader, criterion, device)
scheduler.step()
if val_acc > best_val_acc:
best_val_acc = val_acc
save_checkpoint(model, fine_tune_optimizer, epoch, val_loss, "best_wildlife_model.pt") # Ch.1, Sec.5
print(f"[Fine-tuning] Epoch {epoch+1}: val_acc={val_acc:.4f}")6. Phase 5 — Rigorous Evaluation
Applying the ML Notes' complete classification evaluation toolkit, with particular attention to per-species (not just overall) performance:
from sklearn.metrics import classification_report, confusion_matrix
import numpy as np
model.eval()
all_preds, all_labels = [], []
with torch.no_grad():
for images, labels in val_loader:
images = images.to(device)
outputs = model(images)
_, predicted = torch.max(outputs, 1)
all_preds.extend(predicted.cpu().numpy())
all_labels.extend(labels.numpy())
print(classification_report(all_labels, all_preds, target_names=train_dataset.classes))
# Checking specifically for RARE species performance (Phase 1's stated priority)
cm = confusion_matrix(all_labels, all_preds)
per_class_recall = cm.diagonal() / cm.sum(axis=1)
for class_name, recall in zip(train_dataset.classes, per_class_recall):
print(f"{class_name}: recall = {recall:.3f}")7. Phase 6 — Debugging Along the Way
Applying Chapter 3's debugging discipline — a realistic account of problems this exact project might surface:
Common Issues in a Project Like This
─────────────────────────────────────────
Low recall on RARE species: confirms Phase 1's predicted
imbalance concern — verify
the WeightedRandomSampler
(Phase 2) is configured
correctly, or consider
class-weighted loss (ML
Notes) as an alternative
"Empty" images misclassified camera-trap "empty" images
as a species: often contain partial
animals, motion blur,
or vegetation that
LOOKS animal-like —
may need MORE diverse
"empty" training
examples
Validation accuracy HIGHER check for a data
than training accuracy leakage bug (Module
(unusual!): 4, Ch.3) — e.g.
images from the
SAME camera
location/session
appearing in BOTH
train and val sets
─────────────────────────────────────────
8. Phase 7 — Preparing for Deployment
Connecting directly to the ML Notes' production concerns, applied to a deep learning model specifically:
import torch
# Export the model for a lightweight, dependency-free deployment format
model.eval()
example_input = torch.randn(1, 3, 224, 224).to(device)
traced_model = torch.jit.trace(model, example_input) # TorchScript — deployable WITHOUT the
# full Python training code
traced_model.save("wildlife_classifier_traced.pt")
# The EXACT same batch-vs-real-time consideration from the ML Notes applies here:
# camera-trap images likely arrive in BATCHES (periodic uploads from field cameras),
# favoring a BATCH prediction pipeline over real-time serving9. Where to Go From Here
Module → Technique Used in This Capstone
─────────────────────────────────────────
Module 1 (Foundations) → problem framing, understanding
why DL fits this vision task
Module 2-3 (NN Fundamentals → underlying every layer,
& Training) optimizer, and normalization
used, even inside a
pretrained model
Module 4 (CNNs) → the ResNet50
architecture itself,
data augmentation
Module 7 (Transfer Learning) → the ENTIRE modeling
strategy — feature
extraction, then
fine-tuning
Module 9 (This module) → dataset/DataLoader
patterns,
checkpointing,
debugging,
deployment prep
─────────────────────────────────────────
This capstone deliberately used a pretrained CNN rather than Modules 5-6's sequence architectures or Module 8's generative models — a realistic reflection of how a practitioner chooses tools based on the ACTUAL problem (image classification), rather than using every technique covered just because it exists. Chapter 5 closes out this module — and the whole Deep Learning curriculum — by mapping out exactly where to go next.
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index