Foundations Of Machine Learning
What is Machine Learning?
A Classic, Precise Definition (Tom Mitchell, 1997)
Jr Codex ML Notes
Level: Beginner Prerequisites: None Time to complete: ~20 minutes
Table of Contents
- Defining Machine Learning
- Rules vs Learning — a Direct Comparison
- Why Learn From Data At All?
- Where ML Fits in the Broader AI Landscape
- What Machine Learning Actually Produces: a Model
- Real-World ML, Everywhere
- Summary & Next Steps
1. Defining Machine Learning
Machine Learning (ML) is the field of building systems that improve their performance on a task by learning patterns from data, rather than being explicitly programmed with hand-coded rules for every situation.
A Classic, Precise Definition (Tom Mitchell, 1997)
─────────────────────────────────────────
A program learns from EXPERIENCE (E) with respect to some
TASK (T) and PERFORMANCE MEASURE (P), if its performance
at T, measured by P, improves with experience E.
Example — a spam filter:
T = classifying emails as spam or not
P = percentage of emails correctly classified
E = a dataset of emails, each labeled spam or not
─────────────────────────────────────────
This definition is precise enough to distinguish ML from a merely "smart-looking" program — the defining trait is that performance improves from experience (data), not simply that a program produces good-looking output.
2. Rules vs Learning — a Direct Comparison
# The RULE-BASED approach — a human writes down every condition explicitly
def is_spam_rule_based(email_text: str) -> bool:
spam_keywords = ["win money", "click here now", "free prize", "act now"]
return any(keyword in email_text.lower() for keyword in spam_keywords)
# The MACHINE LEARNING approach — a model LEARNS what makes an email spam
# from thousands of labeled examples, discovering patterns no human explicitly wrote
#
# model.fit(training_emails, training_labels) # LEARN from data
# model.predict(new_email) # APPLY what was learnedWhy This Distinction Matters
─────────────────────────────────────────
Rule-based: works well when the RULES are known, stable,
and few. Fails badly when spammers invent
NEW phrasing the rules never anticipated —
every new pattern requires a HUMAN to notice
it and update the code.
Machine Learning: can discover patterns TOO SUBTLE or NUMEROUS
for a human to hand-code, and can be
RETRAINED on new data as patterns shift —
but requires enough good-quality data to
learn from in the first place.
─────────────────────────────────────────
This is precisely why ML exists as a distinct approach — not because it's inherently "smarter," but because certain problems (recognizing faces, predicting customer churn, translating language) have patterns too complex, numerous, or fast-changing to hand-code exhaustively as fixed rules.
3. Why Learn From Data At All?
Problems Where Hand-Coded Rules Break Down
─────────────────────────────────────────
Image recognition: "a cat" can't be reduced to a
short list of if-statements — the
pixel patterns are far too varied
Predicting customer "will THIS customer churn?"
behavior: depends on dozens of subtly
interacting factors, not a
handful of clean rules
Language translation: grammar and meaning shift
with context in ways that
resist exhaustive rule-writing
Fraud detection: fraud patterns constantly
EVOLVE to evade known rules
─────────────────────────────────────────
In every one of these cases, the pattern exists in the data — it's just too complex or too fluid for a human to write down explicitly. ML's core insight: instead of a human writing the pattern, let an algorithm find it by examining many examples.
4. Where ML Fits in the Broader AI Landscape
The Nested Relationship (full depth in the AI Notes)
─────────────────────────────────────────
Artificial Intelligence (the broadest field — includes
search, logic, planning; NOT all AI involves learning)
│
└── Machine Learning (THIS curriculum — learns from data)
│
└── Deep Learning (ML using neural networks
specifically — a dedicated notes folder)
│
└── NLP/LLMs (deep learning applied to
language specifically — a dedicated
notes folder)
─────────────────────────────────────────
A critical, frequently confused point (also covered in the AI Notes, Module 1): not all AI is machine learning. The AI Notes cover classical, symbolic AI techniques — search algorithms, logical reasoning, planning — that build genuine AI systems without any data-driven learning at all. This entire ML curriculum, by contrast, is specifically about the learning-from-data branch of AI.
5. What Machine Learning Actually Produces: a Model
The output of a machine learning process is a model — a mathematical object that has learned a mapping from inputs to outputs, ready to make predictions on new, unseen data.
# Conceptually, EVERY ML model is a function learned FROM data,
# rather than written BY a human
# A human-written function:
def celsius_to_fahrenheit(c):
return c * 9/5 + 32 # the RELATIONSHIP was already known exactly
# A LEARNED function (conceptual — the actual fitting is covered
# starting in Module 3):
# model = train_model(historical_house_data)
# predicted_price = model.predict(new_house_features)
# — the RELATIONSHIP between features and price was NOT known
# in advance; it was DISCOVERED from thousands of examplesTraining vs Inference — Terminology Worth Knowing Precisely
─────────────────────────────────────────
Training (or "fitting"): the process of learning the
model's parameters FROM data
(Module 3 onward covers this
for specific algorithms)
Inference (or "prediction"): using an ALREADY-TRAINED
model to make a prediction
on NEW data it hasn't seen
─────────────────────────────────────────
6. Real-World ML, Everywhere
ML Is Already Embedded in Everyday Technology
─────────────────────────────────────────
Email: spam filtering, smart reply suggestions
Streaming/Retail: recommendation engines (Netflix, Amazon)
Finance: fraud detection, credit scoring
Healthcare: diagnostic support, drug discovery
Social media: content ranking, ad targeting
Voice assistants: speech recognition (a bridge to
the Deep Learning/NLP Notes)
Search engines: ranking, query understanding
─────────────────────────────────────────
Every one of these examples fits Section 1's formal definition: a specific task, a measurable performance goal, and improvement from experience (data) — this is the lens this entire curriculum will apply to every technique from here forward.
7. Summary & Next Steps
Key Takeaways
- Machine Learning is the field of building systems that improve at a task from experience (data), rather than following exhaustively hand-coded rules.
- ML exists because many real problems (image recognition, fraud detection, translation) have patterns too complex or fast-changing for a human to encode explicitly, but which genuinely exist in data.
- Machine Learning is a specific subfield of the broader field of Artificial Intelligence — not all AI involves learning from data, as the dedicated AI Notes cover in depth.
- The output of ML is a model — training (fitting) learns its parameters from data; inference (prediction) applies the trained model to new data.
Concept Check
- Using Tom Mitchell's definition, identify the T, P, and E for a movie recommendation system.
- Why does hand-coded rule-based spam filtering eventually break down, while an ML approach can adapt?
- Why is "not all AI is machine learning" an important distinction to keep in mind?
Next Chapter
→ Chapter 2: The Three Paradigms
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index