The Generative AI Landscape
What Generative AI Is, and What It Isn't
The important word is sampling. A generative model does not retrieve a stored answer or pick from a fixed list of options. It draws a new sample from a learned
Jr Codex Generative AI Notes
Level: Beginner Prerequisites: None Time to complete: ~15 minutes
Table of Contents
- The One-Sentence Definition
- Discriminative vs Generative
- Why 2022 Was the Inflection Point
- What Generative AI Is Not
- Where This Curriculum Sits
- Summary & Next Steps
1. The One-Sentence Definition
Generative AI is the class of models that produce new content — text, images, audio, video, code — by learning the underlying distribution of their training data and then sampling from it.
The important word is sampling. A generative model does not retrieve a stored answer or pick from a fixed list of options. It draws a new sample from a learned probability distribution, which is why the same prompt can produce a different result twice.
The Core Loop, Stripped Down
─────────────────────────────────────────
TRAINING: see many examples ──► learn P(data)
GENERATION: draw a sample from P(data) ──► new content
"New" means: not in the training set, but plausible
under the same distribution.
─────────────────────────────────────────
2. Discriminative vs Generative
Most models in the Machine Learning Notes were discriminative — they learned a boundary between classes. A generative model learns the classes themselves.
The Distinction
─────────────────────────────────────────
DISCRIMINATIVE learns P(label | data)
"Given this email, is it spam?"
Output: a label, a number, a box
GENERATIVE learns P(data), or P(data | condition)
"Give me an email that looks like spam."
Output: a new example
─────────────────────────────────────────
| Discriminative | Generative | |
|---|---|---|
| Question answered | Which category? | What does one look like? |
| Output | Label or value | New content |
| Examples | Logistic regression, CNN classifier, BERT | GPT, Stable Diffusion, VAEs |
| Evaluation | Accuracy against a known answer | Hard — no known answer |
That last row causes real pain later. A spam classifier is either right or wrong. There is no single correct answer to "write a product description," which is why Module 5 is a full module rather than a footnote.
A Useful Reframe
─────────────────────────────────────────
A generative model is a discriminative model turned
inside out: instead of compressing an input DOWN to a
label, it expands a small input (a prompt, a noise
vector) UP into a full piece of content.
─────────────────────────────────────────
3. Why 2022 Was the Inflection Point
Generative models are not new — the Deep Learning Notes trace autoencoders to the 1980s and GANs to 2014. What changed was not the idea but the usability.
Three Things Converged
─────────────────────────────────────────
1. SCALE
Transformers (DL Notes, Module 6) trained on
internet-scale text crossed the threshold where
output became consistently useful, not merely
grammatical.
2. CONDITIONING
Text became the universal control surface. Before,
you sampled a GAN and took what you got. After, you
described what you wanted in plain English.
3. ACCESS
A capable model became an HTTP request instead of a
six-week training run on hardware you did not own.
─────────────────────────────────────────
Point 2 is the most underrated. The leap that made Stable Diffusion and ChatGPT feel different was not sample quality alone — it was that a non-expert could steer the model in natural language. That shift is why this curriculum spends more time on control (Modules 2 and 3) than on architecture.
4. What Generative AI Is Not
Four misconceptions worth clearing now, because each leads to a predictable engineering mistake later.
Misconception Reality
─────────────────────────────────────────
"It's a search engine It samples a distribution. There is
that writes." no lookup table and no guaranteed
grounding in fact. Grounding is
added with RAG (NLP Notes, Mod.8).
"It stores and copies It stores parameters, not files.
its training data." Verbatim reproduction does happen,
but as memorisation of heavily
repeated data, not retrieval. The
distinction is legally live —
Module 6, Chapter 1.
"Better prompts fix Prompting shifts WHICH part of the
everything." distribution is sampled. It cannot
add knowledge the model lacks, and
cannot make a stochastic process
deterministic.
"Generative means It means novel-under-the-
creative." distribution. Output resembles the
training data by construction — a
strength for style work, a ceiling
for genuine novelty.
─────────────────────────────────────────
5. Where This Curriculum Sits
Generative AI is not a separate branch of AI. It is a capability layered on architectures you have already studied.
The Stack
─────────────────────────────────────────
Agentic AI ACT: plan, remember, use tools
(the NEXT curriculum)
▲
Generative AI PRODUCE: text, images, audio, video
(THIS curriculum — applying the below)
▲
Deep Learning THE ARCHITECTURES: Transformers,
U-Nets, VAEs, GANs, diffusion
▲
Machine Learning THE DISCIPLINE: training, evaluation,
generalisation
─────────────────────────────────────────
This curriculum sits at the application layer. It uses the U-Net without re-deriving convolution, and the Transformer without re-deriving self-attention. When a chapter needs a foundation, it links down the stack rather than repeating it.
6. Summary & Next Steps
Key Takeaways
- A generative model learns
P(data)and produces content by sampling from it — which is why identical prompts give different results. - Discriminative models compress input into a label; generative models expand a small input into full content, and cannot be scored against a single correct answer.
- The 2022 inflection came from scale, natural-language conditioning, and API access — not from a new architecture.
- These models sample rather than retrieve; grounding, determinism, and factual accuracy are things you build around the model, not properties it has.
Concept Check
- Why can a generative model return two different answers to the same prompt, while a spam classifier cannot?
- Which of the three converging factors in Section 3 best explains why non-experts could suddenly use these models?
- A colleague says "the model must be storing the article — it quoted it exactly." What is the more accurate description of what happened?
Next Chapter
→ Chapter 2: The Family of Generative Models
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to Generative AI Index