Foundations Of Agentic AI
Classical Agent Types, Reframed
The AI Notes present five classical agent types. They predate LLMs by decades, and they map onto modern systems precisely — because they classify by what inform
JrCodex·7 min read
Jr Codex Agentic AI Notes
Level: Beginner–Intermediate Prerequisites: Chapter 2: What Makes a System Agentic; AI Notes, Module 5, Chapter 1 Time to complete: ~20 minutes
Table of Contents
- Why an Old Taxonomy Still Earns Its Place
- Simple Reflex Agents
- Model-Based Reflex Agents
- Goal-Based Agents
- Utility-Based Agents
- Learning Agents
- Diagnosing Your Own System
- Summary & Next Steps
1. Why an Old Taxonomy Still Earns Its Place
The AI Notes present five classical agent types. They predate LLMs by decades, and they map onto modern systems precisely — because they classify by what information the agent uses to decide, which is independent of what technology does the deciding.
The Organising Axis
─────────────────────────────────────────
Each type adds ONE thing the previous lacked:
Reflex current input only
Model-based + internal state (memory)
Goal-based + a target and lookahead
Utility-based + a way to RANK outcomes
Learning + improvement from experience
─────────────────────────────────────────
The practical payoff: when an agent misbehaves, this taxonomy usually names the missing capability.
2. Simple Reflex Agents
Classical Modern equivalent
─────────────────────────────────────────
condition-action rules a single LLM call with
on the current percept no history and no tools
─────────────────────────────────────────
def classify_ticket(client, text):
"""A reflex agent. Same input ──► same response. No state, no goal."""
return client.chat.completions.create(
model="gpt-4o-mini", temperature=0,
messages=[{"role": "system", "content": "Reply with exactly one of: "
"BILLING, TECHNICAL, REFUND."},
{"role": "user", "content": text}],
).choices[0].message.content.strip()The Characteristic Failure
─────────────────────────────────────────
A reflex agent cannot handle anything requiring
what came BEFORE. Ask it "what about the second
one?" and it has no idea what the first was.
Levels 1-2 on Chapter 2's spectrum. Fast, cheap,
testable, and correct for a large share of real
tasks.
─────────────────────────────────────────
3. Model-Based Reflex Agents
Adds an internal representation of the world that persists across steps.
Classical Modern equivalent
─────────────────────────────────────────
maintains internal state conversation history,
to handle partial scratchpad, or a
observability working-memory object
─────────────────────────────────────────
class ModelBasedAgent:
def __init__(self, client):
self.client, self.state = client, {} # the internal MODEL of the world
def step(self, observation):
self.state.update(extract_facts(observation)) # update what is believed
return self.client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "system", "content": f"Known state: {self.state}"},
{"role": "user", "content": observation}],
).choices[0].message.contentThe Key Insight for LLM Agents
─────────────────────────────────────────
The conversation history IS the world model, and
it is the ONLY one most agents have.
This is why memory is not a nice-to-have feature
but the difference between two agent classes —
and why Module 3 treats it as foundational rather
than as an optimisation.
─────────────────────────────────────────
4. Goal-Based Agents
Adds an explicit target and the ability to consider whether an action moves toward it.
Classical Modern equivalent
─────────────────────────────────────────
searches ahead for action the ReAct loop —
sequences reaching a goal "have I achieved the
goal, or is another
action needed?"
─────────────────────────────────────────
def goal_based_agent(client, goal, tools, max_steps=10):
messages = [{"role": "system", "content":
f"GOAL: {goal}\nAt each step, state whether the goal is met. "
f"If not, take the action that best advances it."},
{"role": "user", "content": "Begin."}]
for _ in range(max_steps):
reply = client.chat.completions.create(
model="gpt-4o", messages=messages, tools=tools.schemas).choices[0].message
if not reply.tool_calls:
return reply.content # the model judged the GOAL MET
messages.append(reply)
for call in reply.tool_calls:
messages.append({"role": "tool", "tool_call_id": call.id,
"content": str(tools.run(call))})
return NoneThe Distinguishing Property
─────────────────────────────────────────
A goal-based agent can evaluate its own progress.
It has a notion of "not done yet" that a reflex
agent structurally lacks.
This is the threshold at which a system becomes an
agent in the ordinary sense — level 4 on the
spectrum.
─────────────────────────────────────────
5. Utility-Based Agents
Adds preference: not merely whether the goal is met, but how well, so competing options can be ranked.
Classical Modern equivalent
─────────────────────────────────────────
maximises a utility explicit trade-off
function over outcomes criteria in the prompt,
or a scoring function
over candidate plans
─────────────────────────────────────────
UTILITY = """When choosing among approaches, rank by:
1. Correctness — a wrong answer has NEGATIVE value
2. Cost — prefer one cheap tool call over five
3. Latency — prefer fast paths when quality is comparable
Explicitly state the trade-off you are making before acting."""
def utility_based_agent(client, goal, tools):
messages = [{"role": "system", "content": f"GOAL: {goal}\n\n{UTILITY}"}, ...]
# ... same loop; what changed is that the model now has a basis for CHOOSINGWhere This Matters in Practice
─────────────────────────────────────────
Most real agents face several valid routes to the
same goal: search the web, query the database, or
ask the user. All three "work".
Without stated preferences, the model picks
arbitrarily — and arbitrary usually means the most
expensive option, because thoroughness reads as
helpfulness.
Writing down the trade-offs is the cheapest
intervention in this entire curriculum.
─────────────────────────────────────────
6. Learning Agents
Adds improvement from experience.
Classical Modern equivalent
─────────────────────────────────────────
a learning element that three distinct things,
improves the performance often confused:
element from feedback
─────────────────────────────────────────
The Three Kinds of "Learning"
─────────────────────────────────────────
WITHIN A RUN reflection — critique and revise
the current attempt.
Module 4, Chapter 3.
Nothing persists.
ACROSS RUNS write outcomes to long-term
memory and retrieve them later.
Module 3, Chapter 3.
This is where most practical
"learning" lives.
IN THE WEIGHTS fine-tuning on collected
trajectories.
NLP Notes, Module 6. Rare for
agents — expensive, slow, and
usually beaten by better memory.
─────────────────────────────────────────
class LearningAgent:
"""Learns ACROSS runs by recording outcomes and retrieving similar past cases."""
def __init__(self, client, memory): self.client, self.memory = client, memory
def run(self, task):
lessons = self.memory.search(task, k=3) # what happened on similar tasks
result = self.execute(task, prior=lessons)
self.memory.add(task=task, outcome=result.outcome,
lesson=result.what_worked) # write it back for NEXT time
return result7. Diagnosing Your Own System
The taxonomy's real use is diagnostic. Match the symptom to the missing capability.
Symptom Missing Capability
─────────────────────────────────────────
"It forgets what we internal state
discussed two turns ago" ──► model-based (Mod.3)
"It answers, but never a goal and a
checks whether it completion test
actually finished" ──► goal-based
"It always picks the a utility function
slowest, most expensive ──► utility-based
route" (Section 5)
"It makes the same mistake learning across runs
every single time" ──► Module 3, Ch.3
"It retries the identical adaptability
failing action forever" ──► Module 4, Ch.4
─────────────────────────────────────────
The Reframe Worth Keeping
─────────────────────────────────────────
These are not five architectures to choose
between. They are five capabilities that ACCUMULATE.
A production agent is usually all five at once:
reflex for classification, model-based for memory,
goal-based for the loop, utility-based for choosing
tools, learning through stored outcomes.
When one is missing, this table tells you which.
─────────────────────────────────────────
8. Summary & Next Steps
Key Takeaways
- The classical taxonomy classifies by what information drives the decision, so it maps cleanly onto LLM agents despite predating them.
- Each type adds exactly one capability: internal state, then a goal, then a way to rank outcomes, then improvement from experience.
- For LLM agents the conversation history is the world model, which is why memory separates two agent classes rather than being an optimisation.
- Without stated trade-offs an agent chooses arbitrarily among valid routes, usually the most expensive — writing down utility criteria is the cheapest available fix.
Concept Check
- Why does the classical taxonomy still apply to systems built on a technology that did not exist when it was written?
- An agent completes tasks correctly but always uses the slowest tool available. Which capability is missing, and what is the fix?
- Distinguish the three kinds of learning available to an agent, and say which is most practical and why.
Next Chapter
→ Chapter 4: Anatomy of an Agent
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to Agentic AI Index