Foundations Of NLP
What Is NLP & Why Language Is Hard
└── NLP / Large Language Models (THIS curriculum)
Jr Codex NLP & LLM Notes
Level: Beginner Prerequisites: Machine Learning Notes, Deep Learning Notes (especially Module 6: Attention & Transformers) Time to complete: ~20 minutes
Table of Contents
- Where NLP Sits in the Bigger Picture
- What Makes Language Genuinely Hard for Machines
- Ambiguity — the Central Problem
- Common NLP Tasks
- How This Curriculum Is Organized
- Summary & Next Steps
1. Where NLP Sits in the Bigger Picture
Natural Language Processing (NLP) is the field concerned with enabling computers to understand, interpret, and generate human language. Following the AI Notes' landscape framing:
Artificial Intelligence
└── Machine Learning (ML Notes)
└── Deep Learning (DL Notes)
└── NLP / Large Language Models (THIS curriculum)
This curriculum assumes you've completed the Deep Learning Notes, especially Module 6's Transformer architecture — everything here is about applying that general-purpose architecture specifically to language, not re-deriving neural network fundamentals.
2. What Makes Language Genuinely Hard for Machines
Unlike a tabular dataset (ML Notes) or a fixed-resolution image (DL Notes, Module 4), language is:
Why Language Resists Easy Solutions
─────────────────────────────────────────
VARIABLE length: a sentence can be 3 words or 300 —
directly the problem covered in DL
Notes, Module 5, Chapter 1
AMBIGUOUS: the SAME words can mean different
things depending on context
(Section 3)
COMPOSITIONAL: meaning is built from smaller
pieces (morphemes, words,
phrases) combined in
structured, RULE-GOVERNED
(but exception-riddled) ways
CONTEXT-DEPENDENT: the correct interpretation of
a sentence often depends
on information OUTSIDE the
sentence itself — prior
conversation, shared
knowledge, tone
CONSTANTLY EVOLVING: new words, slang, and
usages emerge
continuously — unlike
a fixed set of image
categories (DL Notes,
Module 4)
─────────────────────────────────────────
3. Ambiguity — the Central Problem
Ambiguity deserves special attention, since so much of NLP's history (and this curriculum) is about progressively better ways of resolving it.
Types of Ambiguity, With Examples
─────────────────────────────────────────
Lexical (word-level): "I saw a BAT" — a flying mammal,
or sports equipment?
Syntactic (structure): "I saw the man WITH the
telescope" — who has the
telescope, the speaker or
the man?
Referential: "The city council refused
the protesters a permit
because THEY feared
violence" — who does
"they" refer to? (This
exact example, and its
reverse — "...because
THEY advocated
violence" — is a classic
test of genuine language
UNDERSTANDING, not just
pattern matching)
Pragmatic (implied "Can you pass the
meaning): salt?" — a literal
yes/no question,
but PRAGMATICALLY
a request for action
─────────────────────────────────────────
Resolving ambiguity correctly requires context — and building systems that use context effectively is, in many ways, the story this entire curriculum tells: from simple word-frequency statistics (Chapter 3) to embeddings that capture some contextual meaning (Module 2) to Transformers that model context directly via attention (Module 3, building on DL Notes, Module 6).
4. Common NLP Tasks
A Practical Task Taxonomy
─────────────────────────────────────────
Classification: sentiment analysis, spam detection, topic
labeling — assign a label to a piece
of text (directly the ML Notes'
classification framing, Module 4)
Sequence labeling: part-of-speech tagging, named entity
recognition (NER) — assign a label
to EACH token in a sequence
Generation: machine translation, summarization,
open-ended text generation —
produce NEW text as output
Question answering: given a question (and often a
document), produce an answer
Information retrieval: given a query, find the
most RELEVANT documents
from a large collection
(central to Module 8's
RAG systems)
─────────────────────────────────────────
5. How This Curriculum Is Organized
The Nine Modules Ahead
─────────────────────────────────────────
1. Foundations — this module: what, why, tasks
2. Word Embeddings — representing words as
meaningful vectors
3. Tokenization & — how text becomes model
Pre-training input; how models learn
Foundations from raw text
4. Encoder Models — BERT and understanding
(BERT) tasks
5. Decoder Models — GPT and generation
(GPT) tasks
6. Fine-Tuning & — adapting
Aligning LLMs pretrained
models
7. Prompt Engineering — using
LLMs
effectively
without
training
8. RAG & Agents — grounding
LLMs in
external
knowledge
and actions
9. Evaluation, Production — measuring
& Capstone and
deploying,
end to end
─────────────────────────────────────────
6. Summary & Next Steps
Key Takeaways
- NLP is the field of enabling machines to understand and generate human language, sitting within deep learning and machine learning in the broader AI landscape.
- Language is hard for machines because it is variable-length, ambiguous, compositional, context-dependent, and constantly evolving.
- Lexical, syntactic, referential, and pragmatic ambiguity are the core challenges that progressively better NLP techniques exist to resolve.
- Common NLP tasks fall into classification, sequence labeling, generation, question answering, and information retrieval.
Concept Check
- Why does the "The city council refused the protesters a permit because they feared violence" example illustrate genuine language understanding rather than simple pattern matching?
- Name the four types of ambiguity covered in this chapter, with a one-line example of each.
- Which NLP task category does machine translation fall into, and which does named entity recognition fall into?
Next Chapter
→ Chapter 2: Text Preprocessing Fundamentals
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to NLP & LLM Index