Pandas Primer

Jr Codex Python Notes

Level: Advanced Prerequisites: Chapter 1 Time to complete: ~30 minutes

Why Pandas?
The Series — a Labeled Array
The DataFrame — a Table of Data
Reading Data from Files
Inspecting a DataFrame
Selecting Columns & Rows
Filtering Rows
Adding & Modifying Columns
Handling Missing Data
Grouping & Aggregation
Summary & Next Steps

1. Why Pandas?

NumPy (Chapter 1) is excellent for numeric arrays, but real-world data is rarely just numbers — it has column names, mixed types, and missing values. Pandas builds on top of NumPy to provide labeled, tabular data structures that feel like a spreadsheet or a database table, directly in Python.

import pandas as pd
 
data = {
    "name": ["Alice", "Bob", "Charlie"],
    "age": [25, 30, 35],
    "city": ["Boston", "NYC", "LA"],
}
df = pd.DataFrame(data)
print(df)
#       name  age     city
# 0    Alice   25   Boston
# 1      Bob   30      NYC
# 2  Charlie   35       LA

This directly builds on Module 1's dictionaries and Module 3's CSV/JSON handling — a DataFrame is, in essence, a much more powerful version of the list-of-dicts pattern you've already used.

2. The `Series` — a Labeled Array

A Series is a single column of data — think of it as a NumPy array with labels (an "index") attached to each value.

import pandas as pd
 
ages = pd.Series([25, 30, 35], index=["Alice", "Bob", "Charlie"])
print(ages)
# Alice      25
# Bob        30
# Charlie    35
# dtype: int64
 
print(ages["Bob"])       # 30 — access by LABEL, like a dict
print(ages[0])              # 25 — position-based access still works too
print(ages.mean())            # 30.0 — NumPy-style aggregate methods, inherited from Chapter 1

A DataFrame (Section 3) is essentially a collection of Series objects sharing the same index, one per column.

3. The `DataFrame` — a Table of Data

import pandas as pd
 
data = {
    "name": ["Alice", "Bob", "Charlie", "Diana"],
    "age": [25, 30, 35, 28],
    "salary": [65000, 72000, 58000, 81000],
}
df = pd.DataFrame(data)
print(df)
 
print(df.shape)      # (4, 3) — 4 rows, 3 columns (same shape concept as Chapter 1's arrays)
print(df.columns)      # Index(['name', 'age', 'salary'], dtype='object')
print(df.index)          # RangeIndex(start=0, stop=4, step=1) — default row labels

4. Reading Data from Files

This is where pandas replaces Module 3's manual csv/json modules with something far more convenient for tabular data:

import pandas as pd
 
df = pd.read_csv("people.csv")           # reads an ENTIRE CSV into a DataFrame, one line
df = pd.read_json("people.json")           # same idea, for JSON
df = pd.read_excel("people.xlsx")            # also supports Excel files (needs openpyxl installed)
 
df.to_csv("output.csv", index=False)           # write back out — index=False skips the row-number column
df.to_json("output.json", orient="records")

Module 3's csv/json modules              Pandas
─────────────────────────────           ─────────────────────────────
  Manual row-by-row parsing,               ONE function call reads
  building lists/dicts yourself             the entire file into a table,
                                              with type inference built in
─────────────────────────────           ─────────────────────────────

5. Inspecting a DataFrame

The first thing to run on any new dataset — these methods appear in nearly every real data workflow, including throughout the Machine Learning notes that follow this curriculum:

df.head()          # first 5 rows (pass a number for more/fewer)
df.tail(3)            # last 3 rows
df.info()               # column names, types, non-null counts — a quick health check
df.describe()             # count, mean, std, min/max, quartiles for numeric columns
df.dtypes                   # data type of each column
len(df)                        # number of rows, just like Module 1's len()

print(df.describe())
#              age        salary
# count   4.000000      4.000000
# mean   29.500000  69000.000000
# std     4.203173   9848.857802
# min    25.000000  58000.000000
# ...

6. Selecting Columns & Rows

import pandas as pd
 
df = pd.DataFrame({
    "name": ["Alice", "Bob", "Charlie"],
    "age": [25, 30, 35],
    "salary": [65000, 72000, 58000],
})
 
# Selecting a single column → returns a Series
print(df["age"])
print(df.age)              # equivalent shorthand, IF the column name is a valid Python identifier
 
# Selecting multiple columns → returns a DataFrame (note the DOUBLE brackets)
print(df[["name", "age"]])
 
# Selecting rows by POSITION — .iloc (integer location)
print(df.iloc[0])           # first row, as a Series
print(df.iloc[0:2])           # first 2 rows, as a DataFrame — same slicing rules as Module 1 lists
 
# Selecting rows by LABEL — .loc
print(df.loc[0])              # row labeled 0 (same as iloc here, since the index is default 0,1,2...)
print(df.loc[0, "name"])         # "Alice" — row 0, column 'name'

	`.loc`	`.iloc`
Selects by	Label (index value, column name)	Integer position
Example	`df.loc[0, "name"]`	`df.iloc[0, 0]`

7. Filtering Rows

This is boolean masking (Chapter 1) applied to tabular data — arguably the single most-used pandas operation:

df = pd.DataFrame({
    "name": ["Alice", "Bob", "Charlie", "Diana"],
    "age": [25, 30, 35, 28],
    "salary": [65000, 72000, 58000, 81000],
})
 
adults_over_28 = df[df["age"] > 28]
print(adults_over_28)
 
# Combine conditions with & / | — same rule as NumPy: never use `and`/`or`
high_earners_young = df[(df["age"] < 30) & (df["salary"] > 60000)]
print(high_earners_young)
 
# .isin() — check membership against a list (Module 1's `in`, applied column-wise)
target_names = df[df["name"].isin(["Alice", "Charlie"])]
print(target_names)

8. Adding & Modifying Columns

df["bonus"] = df["salary"] * 0.10                  # new column, vectorized (Chapter 1 style)
print(df)
 
df["seniority"] = df["age"].apply(                      # .apply() runs a function on every value
    lambda age: "Senior" if age >= 30 else "Junior"        # a direct callback to Module 2's lambdas
)
print(df)
 
df["salary"] = df["salary"] + 5000        # modify an existing column, element-wise

.apply() is where Module 2's functions and lambdas directly plug into pandas — any function, not just a lambda, can be passed:

def categorize_age(age):
    if age < 30:
        return "Young"
    elif age < 40:
        return "Middle"
    return "Senior"
 
df["age_group"] = df["age"].apply(categorize_age)

9. Handling Missing Data

Real-world data almost always has gaps — pandas represents a missing value as NaN (Not a Number).

import pandas as pd
import numpy as np
 
df = pd.DataFrame({
    "name": ["Alice", "Bob", "Charlie", "Diana"],
    "age": [25, np.nan, 35, 28],
    "salary": [65000, 72000, np.nan, 81000],
})
 
print(df.isna())            # True/False grid showing where values are missing
print(df.isna().sum())        # count of missing values PER COLUMN
 
df_dropped = df.dropna()        # removes any ROW containing at least one NaN
df_filled = df.fillna(0)          # replaces every NaN with 0
 
df["age"] = df["age"].fillna(df["age"].mean())    # fill with a sensible value — the column's own average

This is only a first look — the Machine Learning notes' Data Preprocessing chapter goes much deeper into missing-data strategy (when to drop vs. impute, and how that choice affects a model).

10. Grouping & Aggregation

.groupby() mirrors Module 5's itertools.groupby (Chapter 4), but is far more convenient — it doesn't require pre-sorting and comes with built-in aggregation methods.

df = pd.DataFrame({
    "department": ["Engineering", "Engineering", "Sales", "Sales", "Sales"],
    "name": ["Alice", "Bob", "Charlie", "Diana", "Eve"],
    "salary": [90000, 85000, 60000, 65000, 62000],
})
 
grouped = df.groupby("department")["salary"].mean()
print(grouped)
# department
# Engineering    87500.0
# Sales          62333.33
 
summary = df.groupby("department").agg(
    avg_salary=("salary", "mean"),
    total_salary=("salary", "sum"),
    headcount=("name", "count"),
)
print(summary)

11. Summary & Next Steps

Key Takeaways

Pandas builds tabular, labeled data structures (Series, DataFrame) on top of NumPy — a direct upgrade over Module 3's manual CSV/JSON handling for anything table-shaped.
pd.read_csv()/read_json() load an entire file in one call; .head(), .info(), .describe() are the standard first steps on any new dataset.
.loc selects by label, .iloc selects by position; boolean masking (df[df["age"] > 28]) is the idiomatic way to filter rows.
.apply() runs any function (including lambdas, Module 2) across a column; .groupby().agg(...) mirrors SQL-style aggregation.

Concept Check

What's the difference between df["age"] and df[["age"]]?
When would you use .loc versus .iloc?
What does df.dropna() do differently from df.fillna(0)?

Next Chapter

→ Chapter 3: Capstone Project

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index