The EDA Workflow

Jr Codex Data Science Notes

Level: Intermediate Prerequisites: Module 2: Data Acquisition & Wrangling Time to complete: ~20 minutes

What is EDA, Really?
Why EDA Comes Before Modeling
The Standard EDA Sequence
First Contact with a New Dataset
Asking Good Questions
Documenting What You Find
Summary & Next Steps

1. What is EDA, Really?

Exploratory Data Analysis (EDA) is the practice of investigating a dataset — through summary statistics, visualizations, and targeted questions — to understand its structure, spot problems, and form hypotheses, before drawing formal conclusions or building a model.

EDA in One Sentence
─────────────────────────────────────────
  "Get to know your data well enough that nothing about
   its structure or quirks surprises you later."
─────────────────────────────────────────

This module is where Modules 1 (statistics) and 2 (wrangling) become tools in service of a real goal: actually understanding a dataset, not just cleaning or describing it in isolation.

2. Why EDA Comes Before Modeling

Skipping straight to modeling on an unexamined dataset is one of the most common (and costly) mistakes in data science.

What Skipping EDA Actually Costs You
─────────────────────────────────────────
  A model trained on data with a data-entry bug
  (e.g. "age" accidentally recorded in months for
  some rows) will silently learn from garbage —
  and the resulting predictions will look plausible
  right up until they're catastrophically wrong.
─────────────────────────────────────────

EDA is how you catch these problems before they contaminate an analysis or a trained model — data quality issues, unexpected distributions, and surprising relationships are far cheaper to fix here than after building on top of them.

3. The Standard EDA Sequence

While EDA is inherently exploratory (not a rigid checklist), most real analyses follow a similar arc:

The EDA Sequence
─────────────────────────────────────────
  1. First contact       → shape, dtypes, head/tail, basic sanity checks (Section 4)
  2. Univariate analysis  → understand EACH column on its own (Chapter 2)
  3. Bivariate/multivariate → understand RELATIONSHIPS between columns (Chapter 3)
  4. Outlier detection      → find and decide what to do with extreme values (Chapter 4)
  5. Summarize findings       → write down what you learned, what's still unclear
─────────────────────────────────────────

This sequence isn't strictly linear in practice — finding an outlier in Step 4 often sends you back to re-examine a univariate distribution from Step 2 — but it's a reliable default order to work through.

4. First Contact with a New Dataset

Before any statistics or plots, get oriented — this is the same checklist introduced briefly in the Python Notes' Pandas Primer, now framed as the deliberate first step of every EDA:

import pandas as pd
 
df = pd.read_csv("customer_data.csv")
 
print(df.shape)          # (rows, columns) — how big is this, really?
print(df.head())            # what do the first few rows actually look like?
print(df.dtypes)              # did numbers get parsed as numbers, dates as strings, etc?
print(df.info())                # dtypes + non-null counts in one view
print(df.describe())              # quick numeric summary (Module 1, Chapter 1)
print(df.isna().sum())               # how much is missing, per column? (Module 2, Chapter 4)
print(df.duplicated().sum())           # any exact duplicate rows? (Module 2, Chapter 4)

# For categorical columns specifically
for col in df.select_dtypes(include="object").columns:
    print(f"\n{col}: {df[col].nunique()} unique values")
    print(df[col].value_counts().head())

Set a rule for yourself: never run a statistical test or build a plot on a column before you've confirmed its dtype, checked for missing values, and glanced at a few actual values — this five-minute habit catches a large fraction of downstream errors.

5. Asking Good Questions

EDA without a direction can wander aimlessly through dozens of plots without producing insight. The best EDA sessions are guided by specific questions, refined as you go.

Starting Questions for ANY New Dataset
─────────────────────────────────────────
  - What does each row REPRESENT? (one customer? one transaction? one day?)
  - What's the TIME RANGE covered, if there's a date column?
  - Are there any columns that seem redundant, or that shouldn't be there?
  - Which column is most likely to be the analysis's actual TARGET
    (the thing you ultimately want to explain or predict)?
  - What would a "typical" row look like, and does that match your
    intuition about the real-world process this data comes from?
─────────────────────────────────────────

# Turning a question into code — "what does a typical row look like?"
print(df.sample(5))         # random sample, often more revealing than .head() (which may show clean, early rows)
print(df.mode().iloc[0])       # the most common value in EACH column

As you answer initial questions, new, more specific ones should naturally emerge — that iterative refinement is the actual "exploratory" part of EDA, and it's exactly what Chapters 2-4 give you the tools to pursue.

6. Documenting What You Find

An EDA session that isn't written down is largely wasted effort — both for collaborators and for your own future self returning to the project weeks later.

A Simple EDA Findings Template
─────────────────────────────────────────
  Dataset: customer_data.csv (10,000 rows, 8 columns)

  Data Quality Issues Found:
   - "income" is 15% missing, appears MAR (correlates with age — Module 2, Ch.4)
   - 3 exact duplicate rows found and removed
   - "signup_date" had 12 rows with an impossible future date — investigating source

  Key Observations:
   - "purchase_amount" is heavily right-skewed (Module 1, Ch.1) — median is
     a better "typical value" than mean
   - Strong positive correlation (r=0.71) between "age" and "income"
   - Customers from "region=West" have a notably higher churn rate — worth
     a closer look in Chapter 3's bivariate analysis
─────────────────────────────────────────

This kind of running log — a plain markdown file, a notebook's markdown cells, or comments in your cleaning script — becomes the backbone of both your final report and the assumptions baked into any model built afterward.

7. Summary & Next Steps

Key Takeaways

EDA is the deliberate process of understanding a dataset's structure, quality, and relationships before drawing conclusions or building models — skipping it risks building on silently broken data.
A reliable default sequence: first contact → univariate → bivariate/multivariate → outliers → written summary.
Always check shape, dtypes, missingness, and a random sample of rows before running any statistics or plots.
Good EDA is question-driven, not aimless — start with a handful of orienting questions and let the data's answers generate more specific follow-ups.
Document findings as you go — an EDA session that isn't written down loses most of its value to collaborators and to your future self.

Concept Check

Why is df.sample(5) sometimes more revealing than df.head() when getting first-contact with a dataset?
What's one real cost of skipping EDA and jumping straight into modeling?
Why is EDA described as "question-driven" rather than a rigid, one-size-fits-all checklist?

Next Chapter

→ Chapter 2: Univariate Analysis

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index