The Machine Learning Workflow

Jr Codex ML Notes

Level: Beginner–Intermediate Prerequisites: Chapter 2 Time to complete: ~25 minutes

Why a Workflow, Not Just an Algorithm
Step 1: Problem Definition
Step 2: Data Collection
Step 3: Exploratory Data Analysis
Step 4: Data Preparation
Step 5: Model Selection & Training
Step 6: Evaluation
Step 7: Deployment & Monitoring
The Workflow Is a Loop
Summary & Next Steps

1. Why a Workflow, Not Just an Algorithm

A common beginner misconception: machine learning is mostly about choosing and running an algorithm. In practice, the algorithm (Modules 3-6) is often the smallest part of a real ML project — most of the work happens before and after it. This chapter maps the full process; later modules deep-dive into individual steps.

Where Effort Actually Goes in a Real ML Project
─────────────────────────────────────────
  Problem definition + data collection + EDA + preparation:
    often 70-80% of total project time

  Model selection + training:
    often surprisingly QUICK, once data is ready — a few
    lines of code with modern libraries

  Evaluation + deployment + monitoring:
    ongoing, ostensibly "after the fact" but often just as
    much work as everything before it
─────────────────────────────────────────

This directly echoes the Data Science Notes' CRISP-DM chapter — that framework generalizes across any data-driven project; this chapter specializes it specifically for supervised/unsupervised ML.

2. Step 1: Problem Definition

Before writing any code, get precise about what's actually being predicted or discovered, and how success will be measured — directly connecting to Chapter 2's paradigm map.

Questions to Answer Before Touching Data
─────────────────────────────────────────
  - Is this SUPERVISED (do we have labeled outcomes), or
    UNSUPERVISED (Chapter 2)?
  - If supervised: REGRESSION or CLASSIFICATION?
  - What's the PERFORMANCE MEASURE that actually matters for
    the business/research question? (Chapter 1's P — this is
    often more nuanced than "just be accurate," a theme
    revisited heavily in Module 4's evaluation chapter)
  - What does a "good enough" model look like, concretely?
─────────────────────────────────────────

Skipping this step is a common, costly mistake — building a highly accurate model that answers the wrong question, or optimizes the wrong metric, wastes enormous effort. This mirrors the Data Science Notes' "Business Understanding" phase precisely.

3. Step 2: Data Collection

Where Training Data Comes From (Full Depth in the Data Science Notes)
─────────────────────────────────────────
  - Existing company databases (Data Science Notes, Module 2, Ch.1-2)
  - Public datasets (Kaggle, UCI ML Repository, government open data)
  - APIs and web scraping (Data Science Notes, Module 2, Ch.1)
  - Manually labeled data (human annotators labeling examples —
    often necessary for supervised learning when no labels
    exist naturally)
─────────────────────────────────────────

A Critical Early Question: Is There Enough Labeled Data?
─────────────────────────────────────────
  Supervised learning REQUIRES labels — if your problem needs
  supervised learning (Chapter 2) but labels don't exist yet,
  labeling data becomes a real, often expensive project
  requirement, not an afterthought.
─────────────────────────────────────────

4. Step 3: Exploratory Data Analysis

This step is covered in complete depth in the Data Science Notes' EDA module — univariate/bivariate analysis, outlier detection, and the full EDA workflow apply directly and identically here. Worth restating why it matters specifically for ML:

What EDA Reveals BEFORE Modeling
─────────────────────────────────────────
  - Missing data patterns (determines Module 2's cleaning strategy)
  - Class imbalance (Module 2, Ch.5 — critical for classification)
  - Feature distributions and outliers (informs Module 2's scaling
    and Module 3-4's algorithm choice — some algorithms are more
    sensitive to outliers than others)
  - Correlations between features (relevant to Module 6's
    dimensionality reduction, and to detecting redundant features)
─────────────────────────────────────────

Skipping EDA and jumping straight to model training is one of the most common mistakes beginners make — a model trained on unexamined data will faithfully learn whatever quirks (including bugs, outliers, or leakage) exist in that data, with no way to distinguish "real pattern" from "data problem."

5. Step 4: Data Preparation

Covered in full depth in Module 2 — cleaning, encoding categorical variables, scaling numeric features, feature engineering, and handling imbalanced data. This is the step where raw, EDA-examined data becomes genuinely ready for an algorithm to consume.

A Quick Preview of Module 2's Content
─────────────────────────────────────────
  Raw data:            {"city": "Boston", "income": 65000, "age": 34}

  Model-ready data:       [1, 0, 0, 0,    0.42,      0.31]
                            (one-hot        (scaled    (scaled
                             encoded         income)     age)
                             city)
─────────────────────────────────────────

6. Step 5: Model Selection & Training

Once data is ready, choosing and training an algorithm (Modules 3-6, depending on the paradigm from Chapter 2) is often the most mechanically straightforward step — most modern ML libraries reduce this to a few lines of code.

# The GENERAL shape of this step, regardless of which specific
# algorithm from Modules 3-6 you end up choosing:
 
# 1. Split data (Chapter 5 covers this properly)
# X_train, X_test, y_train, y_test = train_test_split(X, y)
 
# 2. Choose and train a model
# model = SomeAlgorithm()
# model.fit(X_train, y_train)
 
# 3. Make predictions
# predictions = model.predict(X_test)

The real skill here isn't memorizing library syntax (which is genuinely simple) — it's knowing which algorithm fits the problem's paradigm, data size, interpretability needs, and performance requirements, a judgment this entire curriculum builds toward across Modules 3-7.

7. Step 6: Evaluation

Covered properly starting in Module 1, Chapter 5 (train/test basics) and in depth at the end of Modules 3-4 (regression and classification metrics respectively). The core question: does the trained model actually perform well on data it hasn't seen, not just the data it was trained on?

Why "Accuracy on Training Data" Is Nearly Meaningless
─────────────────────────────────────────
  A model can achieve NEAR-PERFECT accuracy on the exact data
  it was trained on, while performing TERRIBLY on new data —
  this is OVERFITTING, the single most important failure mode
  in all of machine learning, covered fully in Chapter 4.
─────────────────────────────────────────

8. Step 7: Deployment & Monitoring

Covered in depth in Module 9 — getting a validated model into a real system where it makes predictions on genuinely new, live data, and watching for degrading performance over time.

Why This Step Is Often Underestimated
─────────────────────────────────────────
  A model that performs beautifully in a research notebook can
  fail in production for reasons that have NOTHING to do with
  the algorithm itself: data pipeline bugs, the real world's
  data DRIFTING away from the training data's patterns over
  time, or infrastructure that can't serve predictions fast
  enough — all covered in Module 9.
─────────────────────────────────────────

9. The Workflow Is a Loop

Just like the Data Science Notes' CRISP-DM chapter emphasized, this workflow rarely proceeds cleanly in one pass — evaluation routinely sends you back to earlier steps.

Common Loop-Backs
─────────────────────────────────────────
  Evaluation reveals POOR performance
    → back to Step 4 (more feature engineering) or
       Step 5 (try a different algorithm/Module 7's tuning)

  Evaluation reveals the model is TECHNICALLY good but doesn't
  actually solve the business problem
    → back to Step 1 (the problem was mis-defined)

  Monitoring in production reveals performance DEGRADING
  over time (Module 9's "model drift")
    → back to Step 2 (collect fresh data) and Step 6
      (retrain and re-evaluate)
─────────────────────────────────────────

10. Summary & Next Steps

Key Takeaways

The ML workflow spans problem definition, data collection, EDA, data preparation, model training, evaluation, and deployment/monitoring — the algorithm itself is often the smallest, quickest part.
Problem definition (paradigm, metric, "good enough" threshold) must happen before touching data — skipping it risks solving the wrong problem well.
EDA and data preparation (covered fully in the Data Science Notes and this curriculum's Module 2) typically consume the majority of real project time.
The workflow loops — evaluation and production monitoring routinely send a project back to earlier steps, and that's normal, healthy iteration rather than failure.

Concept Check

Why is "accuracy on the training data" a poor measure of whether a model actually works?
What kind of problem might cause you to loop all the way back to Step 1 (problem definition) after evaluation?
Name two reasons a model might fail in production despite performing well during development.

Next Chapter

→ Chapter 4: Bias-Variance Tradeoff, Overfitting & Underfitting

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index