The Reasoning Core
The LLM as a Reasoning Engine
Strip away the framing and the planner performs one operation, repeatedly:
JrCodex·7 min read
Jr Codex Agentic AI Notes
Level: Intermediate Prerequisites: Module 1, Chapter 4: Anatomy of an Agent Time to complete: ~20 minutes
Table of Contents
- What the Planner Actually Does
- Context Is the Working Memory
- What It Reasons About Well
- What It Reasons About Badly
- Context Degradation Over a Run
- Designing Around the Limits
- Summary & Next Steps
1. What the Planner Actually Does
Strip away the framing and the planner performs one operation, repeatedly:
The Only Operation
─────────────────────────────────────────
INPUT everything that has happened so far,
serialised as text
OUTPUT the single next action, as text
There is no persistent internal state between
calls, no accumulating "understanding", and no
memory that is not in the input.
Every step reconstructs the agent's entire
understanding from scratch, from the transcript.
─────────────────────────────────────────
Why This Framing Matters
─────────────────────────────────────────
It means an agent's competence is a function of
what is IN THE CONTEXT, not of how capable the
model is in general.
A frontier model with a badly assembled context
will reason worse than a small model with a clean
one. Most agent debugging turns out to be context
debugging.
─────────────────────────────────────────
2. Context Is the Working Memory
The context window is the agent's entire cognitive workspace, and it fills up predictably.
What Occupies It, Step by Step
─────────────────────────────────────────
Step 0 system prompt + tool schemas ~2,000
Step 1 + reasoning + call + result ~3,500
Step 2 + reasoning + call + result ~5,200
Step 3 + reasoning + call + result ~9,800
(one verbose tool result)
Step 4 + reasoning + call + result ~11,000
...
Step 12 ~48,000
Growth is superlinear in practice, because tool
results vary wildly in size and the largest one
dominates everything after it.
─────────────────────────────────────────
def context_report(messages, enc):
"""Know where the tokens actually go before optimising anything."""
by_role = {}
for m in messages:
content = m.get("content") or ""
by_role[m["role"]] = by_role.get(m["role"], 0) + len(enc.encode(str(content)))
total = sum(by_role.values())
for role, n in sorted(by_role.items(), key=lambda kv: -kv[1]):
print(f" {role:10} {n:7,} {100*n/total:5.1f}%")
return total
# Typical finding on a long run:
# tool 38,420 79.4% ← almost always the answer
# assistant 7,110 14.7%
# system 2,050 4.2%
# user 840 1.7%Nearly every context problem is a tool-result problem, which is why the perception layer from Module 1, Chapter 4 is worth real attention.
3. What It Reasons About Well
Genuine Strengths
─────────────────────────────────────────
TOOL SELECTION choosing among 5-15 well
described tools, given a clear
goal. Reliable.
ARGUMENT filling a schema from context,
CONSTRUCTION including reformatting values
into the shape a tool wants.
ERROR reading an error message and
INTERPRETATION choosing a different approach —
the single most useful agentic
behaviour, and why Module 1's
errors-as-observations rule pays
off so heavily.
SYNTHESIS combining several tool results
into a coherent answer.
SHORT-HORIZON "given this result, what next?"
PLANNING over 2-5 steps.
─────────────────────────────────────────
4. What It Reasons About Badly
Structural Weaknesses
─────────────────────────────────────────
LONG-HORIZON beyond ~10 steps, the model
PLANNING loses the thread of the
original goal. Fix: explicit
plan state (Module 4).
ARITHMETIC AND still text prediction. Fix: a
PRECISE COUNTING calculator tool, always.
KNOWING WHAT IT it cannot reliably report
DOES NOT KNOW uncertainty. It will act
confidently on a wrong premise.
NEGATIVE "these three tools all failed,
CONCLUSIONS so the data does not exist" is
a leap it rarely makes; it
keeps trying instead.
SELF-ASSESSMENT asked "did you complete the
task?", it says yes far more
often than it should.
─────────────────────────────────────────
The Two With the Largest Blast Radius
─────────────────────────────────────────
UNRELIABLE SELF-ASSESSMENT is why agent evaluation
cannot ask the agent (Module 7), and why a
completion check must be external.
NO SENSE OF UNCERTAINTY is why irreversible actions
need a human gate (Module 8) rather than a
confidence threshold — there is no calibrated
confidence to threshold on.
─────────────────────────────────────────
5. Context Degradation Over a Run
Agent quality does not decline gracefully with context length. It declines in a specific, recognisable pattern.
The Failure Curve
─────────────────────────────────────────
Steps 1-4 sharp. Goal clear, tool choice good.
Steps 5-9 the ORIGINAL GOAL drifts toward the
middle of the context — exactly the
"lost in the middle" position (Gen AI
Notes, Module 2, Chapter 4).
The agent begins optimising for the
LAST tool result rather than the task.
Steps 10+ it re-runs tools it already ran,
because their results have scrolled
into the low-attention region and
are effectively invisible.
─────────────────────────────────────────
The Tell
─────────────────────────────────────────
A repeated tool call with identical arguments is
almost never a reasoning failure. It is a CONTEXT
failure — the earlier result is still present but
no longer salient.
Diagnose it by checking whether the result is in
the transcript. It usually is.
─────────────────────────────────────────
6. Designing Around the Limits
Each limit has a concrete mitigation, and none requires a better model.
GOAL_ANCHOR = """ORIGINAL GOAL: {goal}
Progress so far:
{completed}
Remaining:
{remaining}
Re-read the goal above before choosing your next action."""
def anchored_step(client, goal, state, messages, tools):
"""Re-state the goal at the END of the context, every step."""
anchor = {"role": "user", "content": GOAL_ANCHOR.format(
goal=goal,
completed="\n".join(f" ✓ {s}" for s in state.done) or " (nothing yet)",
remaining="\n".join(f" · {s}" for s in state.todo) or " (unknown)")}
return client.chat.completions.create(
model="gpt-4o",
messages=[*messages, anchor], # LAST position = highest attention
tools=tools,
).choices[0].messagedef dedupe_calls(state, call):
"""Prevent the repeated-call failure mechanically rather than by prompting."""
key = (call.function.name, call.function.arguments)
if key in state.seen:
return {"note": "You already called this tool with these exact arguments. "
"The earlier result was:\n" + state.seen[key] +
"\nUse it, or take a DIFFERENT action."}
return None # not a repeat — proceed normallyThe Four Mitigations
─────────────────────────────────────────
GOAL DRIFT re-anchor the goal at the END of
the context every step
REPEATED CALLS deduplicate in the EXECUTOR and
return the cached result with a
nudge — do not rely on the model
noticing
CONTEXT BLOAT truncate and summarise tool
results in perception (Module 3)
FALSE COMPLETION verify against an external check,
never against the agent's own
claim (Module 7)
─────────────────────────────────────────
7. Summary & Next Steps
Key Takeaways
- The planner has no state between calls; it reconstructs its entire understanding from the transcript every step, so agent competence is a property of the assembled context.
- Tool results typically consume 75–80% of an agent's context, making perception the highest-leverage place to optimise.
- The model is strong at tool selection, argument construction and error interpretation; weak at long-horizon planning, arithmetic, negative conclusions and self-assessment.
- Repeated identical tool calls signal context failure rather than reasoning failure — fix them in the executor, and re-anchor the goal at the end of the context.
Concept Check
- Why can a frontier model perform worse than a smaller one on the same agentic task?
- An agent calls
get_user(id=42)for the third time at step 11. What is the mechanism, and why is prompting a poor fix? - Which two weaknesses have the largest downstream consequences for how agents must be evaluated and controlled?
Next Chapter
→ Chapter 2: Prompting Patterns for Agents
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to Agentic AI Index