Data Acquisition And Wrangling
Feature Engineering Basics
A feature is a single measurable column of input data used for analysis or modeling. Feature engineering is creating new, more useful features from the raw ones
Jr Codex Data Science Notes
Level: Intermediate Prerequisites: Chapter 4 Time to complete: ~25 minutes
Table of Contents
- What is a Feature, and Why Engineer One?
- Creating Derived Features
- Binning — Turning Continuous Values into Categories
- Encoding Categorical Variables
- Scaling Numeric Features
- Extracting Features from Dates
- Extracting Features from Text
- Summary & Next Steps
1. What is a Feature, and Why Engineer One?
A feature is a single measurable column of input data used for analysis or modeling. Feature engineering is creating new, more useful features from the raw ones you started with — often the single highest-leverage activity in a data science project, more impactful than switching algorithms in the Machine Learning notes.
Raw Data Engineered Features
───────────────────────────── ─────────────────────────────
signup_date: "2024-01-15" days_since_signup: 543
purchase_amount: 45.00 is_weekend_purchase: True
purchase_amount_bracket: "medium"
───────────────────────────── ─────────────────────────────
This chapter previews the topic; the Machine Learning notes' Data Preprocessing chapter goes further into feature engineering specifically for model training.
2. Creating Derived Features
The simplest form: combine or transform existing columns into something more directly useful.
import pandas as pd
df = pd.DataFrame({
"first_name": ["Alice", "Bob"],
"last_name": ["Smith", "Jones"],
"price": [100, 250],
"quantity": [3, 1],
})
df["full_name"] = df["first_name"] + " " + df["last_name"]
df["total_cost"] = df["price"] * df["quantity"]
df["price_per_unit"] = df["total_cost"] / df["quantity"]
# Ratio features are often more informative than either raw column alone
df["is_bulk_order"] = df["quantity"] >= 3The best derived features usually come from domain knowledge — knowing that "price per unit" or "days since last purchase" matters requires understanding the actual business problem, not just the columns available.
3. Binning — Turning Continuous Values into Categories
Binning groups a continuous numeric variable into discrete buckets — useful for simplifying a variable, handling non-linear effects, or making a value more interpretable in a report.
import pandas as pd
df = pd.DataFrame({"age": [22, 35, 45, 19, 62, 51, 28]})
# Fixed-width bins
df["age_group"] = pd.cut(
df["age"],
bins=[0, 25, 40, 60, 100],
labels=["18-25", "26-40", "41-60", "60+"],
)
print(df)
# Equal-FREQUENCY bins (quantiles) — each bin gets roughly the same NUMBER of records
df["age_quartile"] = pd.qcut(df["age"], q=4, labels=["Q1", "Q2", "Q3", "Q4"])cut() vs qcut()
─────────────────────────────────────────
pd.cut() → bins of EQUAL WIDTH (e.g. every 20 years) — some bins may end up
with far more records than others
pd.qcut() → bins with roughly EQUAL COUNT of records — bin WIDTHS may vary
─────────────────────────────────────────
4. Encoding Categorical Variables
Most statistical and ML tools require numeric input — encoding converts categories (strings) into numbers, in a way that preserves their meaning as much as possible.
import pandas as pd
df = pd.DataFrame({
"size": ["small", "medium", "large", "medium", "small"],
"color": ["red", "blue", "green", "red", "blue"],
})
# One-hot encoding — one binary column PER category, no false ordering implied
one_hot = pd.get_dummies(df["color"], prefix="color")
print(one_hot)
# color_blue color_green color_red
# 0 False False True
# 1 True False False
# ...
# Ordinal encoding — for categories that DO have a natural order
size_order = {"small": 0, "medium": 1, "large": 2}
df["size_encoded"] = df["size"].map(size_order)
print(df[["size", "size_encoded"]])One-Hot vs Ordinal Encoding — When to Use Which
─────────────────────────────────────────
One-hot: NO natural order (color, city, product category)
→ creates multiple binary columns, avoids implying a false ranking
Ordinal: a REAL, meaningful order exists (small < medium < large)
→ a single numeric column, preserving the order relationship
─────────────────────────────────────────
Common mistake: ordinal-encoding a variable with no real order (e.g. {"red": 0, "blue": 1, "green": 2}) silently implies green > blue > red, which is meaningless and can mislead any statistical method or model that assumes numeric relationships matter.
5. Scaling Numeric Features
Different numeric columns often live on wildly different scales — age (0-100) versus income ($0-$500,000). Scaling puts them on a comparable footing.
import pandas as pd
df = pd.DataFrame({
"age": [25, 45, 35, 50, 23],
"income": [45000, 95000, 65000, 120000, 38000],
})
# Min-Max scaling — rescales to a fixed [0, 1] range
df["age_scaled"] = (df["age"] - df["age"].min()) / (df["age"].max() - df["age"].min())
# Standardization (Z-score) — rescales to mean=0, std_dev=1 (directly using Module 1's stats)
df["income_standardized"] = (df["income"] - df["income"].mean()) / df["income"].std()
print(df)Min-Max Scaling Standardization (Z-score)
───────────────────────────── ─────────────────────────────
Result range: [0, 1] Result: mean=0, std dev=1
Formula: (x - min) / (max - min) Formula: (x - mean) / std_dev
Sensitive to outliers (min/max Less sensitive to outliers than
can be extreme values) min-max, but not immune
───────────────────────────── ─────────────────────────────
Why this matters: many statistical/ML methods (distance-based methods, gradient-based optimization) implicitly treat larger-scale numbers as "more important" unless everything is scaled comparably first — this is a recurring theme picked back up in the Machine Learning notes' preprocessing chapter.
6. Extracting Features from Dates
Building directly on Chapter 3's pd.to_datetime() — a single date column often hides several separately useful features.
import pandas as pd
df = pd.DataFrame({"order_date": pd.to_datetime(["2024-01-15", "2024-06-22", "2024-12-25"])})
df["year"] = df["order_date"].dt.year
df["month"] = df["order_date"].dt.month
df["day_of_week"] = df["order_date"].dt.day_name()
df["is_weekend"] = df["order_date"].dt.dayofweek >= 5 # Saturday=5, Sunday=6
df["quarter"] = df["order_date"].dt.quarter
df["days_since_order"] = (pd.Timestamp.now() - df["order_date"]).dt.days
print(df)These derived features (day of week, weekend flag, quarter) often reveal patterns a raw timestamp never would — e.g. "orders spike on weekends" is invisible until is_weekend exists as its own column to group/filter by.
7. Extracting Features from Text
A brief preview — full text feature extraction (TF-IDF, embeddings) is covered in depth in the NLP/LLM notes, but a few simple, high-value features are worth knowing here.
import pandas as pd
df = pd.DataFrame({
"review": [
"This product is absolutely amazing!",
"Terrible. Do not buy.",
"It's okay, nothing special.",
]
})
df["review_length"] = df["review"].str.len()
df["word_count"] = df["review"].str.split().str.len()
df["has_exclamation"] = df["review"].str.contains("!")
df["contains_negative_word"] = df["review"].str.lower().str.contains("terrible|bad|awful")
print(df)Even these simple counts (length, word count, punctuation presence) are legitimate, useful features on their own — full semantic understanding of text is a much deeper topic, picked up later in this curriculum's NLP/LLM notes.
8. Summary & Next Steps
Key Takeaways
- Feature engineering — creating new, more useful columns from raw data — is often the highest-leverage activity in a data science project.
- Binning (
pd.cut/pd.qcut) turns continuous variables into interpretable categories; choose equal-width vs equal-frequency bins based on the goal. - One-hot encode categories with no natural order; ordinal-encode only when a real order exists — mixing these up silently implies false rankings.
- Scaling (min-max or standardization) puts differently-scaled numeric columns on comparable footing — important for many downstream statistical and ML methods.
- Dates and text both hide multiple simple, high-value derived features (day of week, weekend flag, word count, punctuation) beyond their raw form.
Module 2 Complete — Next Module
You can now acquire data from real sources, query it with SQL, clean it, handle missing/duplicate records deliberately, and engineer useful features. Module 3 puts all of this together into a structured process for actually exploring and understanding a dataset — Exploratory Data Analysis.
Concept Check
- Why is ordinal-encoding a variable like
color(with no real order) a common and misleading mistake? - What's the difference between
pd.cut()andpd.qcut()? - Why might "day of week" extracted from a date column reveal a pattern the raw timestamp never would?
Next Chapter
→ Module 3: Exploratory Data Analysis
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index