Foundations Of AI
History of AI
AI has gone through repeated cycles of hype and disappointment ("AI winters") — understanding this history explains why the field is organized the way it is tod
Jr Codex AI Notes
Level: Beginner Prerequisites: Chapter 1 Time to complete: ~20 minutes
Table of Contents
- Why History Matters for Learning AI
- The Birth of AI (1940s-1956)
- The Golden Years (1956-1974)
- The First AI Winter (1974-1980)
- Expert Systems Boom & Bust (1980-1993)
- The Statistical/ML Turn (1990s-2010)
- The Deep Learning Era (2012-Present)
- Lessons from AI's Boom-Bust Cycles
- Summary & Next Steps
1. Why History Matters for Learning AI
AI has gone through repeated cycles of hype and disappointment ("AI winters") — understanding this history explains why the field is organized the way it is today, and why this curriculum covers both classical techniques (Modules 2-3, largely from the "Golden Years") and modern deep learning (a separate dedicated notes folder) rather than treating AI as a single, uniform subject.
The Full Timeline, At a Glance
─────────────────────────────────────────
1940s-56 Birth: early ideas, Turing's paper, the Dartmouth Workshop
1956-74 Golden Years: huge optimism, early wins, big funding
1974-80 First AI Winter: overpromising meets underdelivering
1980-93 Expert Systems boom, then a second bust
1990s-2010 The statistical/ML turn — quieter, steadier progress
2012-now Deep Learning era — massive capability jump
─────────────────────────────────────────
2. The Birth of AI (1940s-1956)
Key Milestones
─────────────────────────────────────────
1943 McCulloch & Pitts propose a mathematical model of a
neuron — the conceptual root of neural networks (DL notes)
1950 Alan Turing publishes "Computing Machinery and
Intelligence," proposing the Turing Test (Ch.1)
1956 The Dartmouth Workshop — the term "Artificial
Intelligence" is coined by John McCarthy; widely
considered the FOUNDING event of AI as a field
─────────────────────────────────────────
The Dartmouth Workshop's proposal captured the field's founding optimism: organizers believed that "every aspect of learning...can in principle be so precisely described that a machine can be made to simulate it" — and that meaningful progress could be made by a small group over a single summer. This optimism (and its eventual collision with reality) is a pattern that recurs throughout AI's history.
3. The Golden Years (1956-1974)
Early successes generated enormous confidence — narrow demonstrations were mistaken for signs of imminent general intelligence.
Notable Achievements
─────────────────────────────────────────
1957 The General Problem Solver (GPS) — an early system
using SEARCH (Module 2) to solve puzzles
1958 John McCarthy invents LISP, AI's dominant
programming language for decades
1965 ELIZA — a simple pattern-matching chatbot that
convinced some users it understood them
(an early, informal precursor to the Turing
Test debate from Chapter 1)
1969 Shakey the robot — combined perception, planning,
and action in a physical robot
─────────────────────────────────────────
Funding poured in (particularly from DARPA in the US), and prominent researchers made bold, now-famous predictions — including claims that human-level AI was only a decade or two away. These predictions would prove badly premature, setting up the first winter.
4. The First AI Winter (1974-1980)
What Went Wrong
─────────────────────────────────────────
- Computers of the era had FAR too little memory and
processing power for the ambitions researchers had
- Many early techniques didn't SCALE — they worked on toy
problems but collapsed on realistic, larger ones
(a recurring theme, revisited in Module 2's search chapters)
- The 1973 Lighthill Report (UK) sharply criticized AI's
lack of practical progress, leading to major funding cuts
- DARPA also significantly cut AI funding in the US following
similarly disappointing results
─────────────────────────────────────────
"AI Winter" became the term for this pattern: a period of reduced funding and interest following a period of unmet, overhyped expectations — this first winter would not be the last.
5. Expert Systems Boom & Bust (1980-1993)
The Second Wave
─────────────────────────────────────────
1980s EXPERT SYSTEMS (Module 3's rule-based AI) become
commercially successful — systems like MYCIN
(medical diagnosis) and XCON (configuring computer
systems) demonstrate real business value
1980s Japan's "Fifth Generation Computer" project and
similar government initiatives pour money back
into AI research worldwide
Late 1980s Expert systems prove EXPENSIVE to build and
maintain, brittle when facing situations outside
their narrow, hand-coded rules — a second wave
of disillusionment follows
1987-93 The SECOND AI Winter — specialized "Lisp
Machine" hardware companies collapse, funding
contracts again
─────────────────────────────────────────
Expert systems (covered in Module 3, Chapter 6) demonstrated a recurring lesson: hand-coded knowledge works well within a narrow domain but doesn't generalize — a limitation that would later motivate the shift toward learning from data.
6. The Statistical/ML Turn (1990s-2010)
A Quieter, More Durable Era
─────────────────────────────────────────
1990s Researchers increasingly shift from hand-coded RULES
toward STATISTICAL and PROBABILISTIC methods
(Module 4's Bayesian networks trace to this era)
1997 IBM's Deep Blue defeats world chess champion Garry
Kasparov — a triumph of SEARCH (Module 2) and
specialized hardware, not learning
Early 2000s Machine learning (a separate notes folder in
this curriculum) matures as its own
discipline — support vector machines, random
forests, and other now-classic ML algorithms
become standard tools
2006 Geoffrey Hinton and colleagues popularize
techniques for training DEEP neural networks
effectively — planting the seed for the next era
─────────────────────────────────────────
This period avoided a third full "winter" — progress was steadier and less over-promised, partly because the field had learned from its earlier boom-bust cycles (Section 8).
7. The Deep Learning Era (2012-Present)
The Modern Explosion
─────────────────────────────────────────
2012 AlexNet dramatically wins the ImageNet competition
using deep convolutional neural networks — the
moment widely credited with igniting the modern
deep learning boom (full depth in the DL notes)
2016 DeepMind's AlphaGo defeats a world champion Go player
— combining deep learning WITH search (Module 2)
and reinforcement learning (Module 5)
2017 The "Transformer" architecture is introduced — the
foundation of virtually every modern large language
model (full depth in the NLP/LLM notes)
2020-Present GPT-3, GPT-4, Claude, and other large language
models demonstrate startlingly fluent,
general-purpose language capabilities
─────────────────────────────────────────
Why this era has (so far) avoided a third winter: unlike prior booms, deep learning's gains have been backed by dramatic, measurable improvements on real, hard benchmarks (image recognition, translation, game-playing, and more), combined with genuine commercial products reaching billions of users.
8. Lessons from AI's Boom-Bust Cycles
Recurring Patterns Worth Remembering
─────────────────────────────────────────
1. Narrow success gets OVER-GENERALIZED into predictions of
imminent general intelligence — this happened in the 1960s,
the 1980s, and arguably continues to be debated today
2. Techniques that work on TOY problems often fail to SCALE —
a caution directly relevant to Module 2's search algorithms,
which can explode in computational cost as problems grow
3. Hand-coded knowledge (expert systems) is brittle outside
its narrow domain — a key motivation for the entire field
of machine learning that followed
4. Genuine, durable progress tends to follow measurable,
benchmarked improvements on real tasks — not just
impressive demos or bold predictions
─────────────────────────────────────────
These lessons directly inform how this curriculum is structured — Module 2's search chapters, for instance, spend real time on why certain algorithms scale poorly, not just how they work in the abstract.
9. Summary & Next Steps
Key Takeaways
- AI's history moves in cycles: the 1956 Dartmouth Workshop's optimism, the Golden Years' early wins, two separate "AI winters" driven by overpromising, and the current deep learning era beginning around 2012.
- Expert systems (1980s) demonstrated that hand-coded knowledge works narrowly but doesn't generalize — a key motivation for the shift toward learning-based methods.
- The current deep learning/LLM era has so far avoided a third winter by grounding progress in measurable benchmarks and real commercial products, not just bold predictions.
- Recurring lesson across every cycle: narrow demonstrations get over-generalized into premature claims of general intelligence — a pattern worth watching for even today.
Concept Check
- What caused the first AI Winter (1974-1980), and what's the parallel to the second one in the late 1980s?
- Why did expert systems eventually fall out of favor despite early commercial success?
- What's different about the current deep learning era that has (so far) prevented a third AI winter?
Next Chapter
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index