The Reasoning Core
Prompting Patterns for Agents
The NLP Notes covered prompting for a single response. An agent prompt is a different object: it is evaluated at every step of a run, against a context that cha
JrCodex·8 min read
Jr Codex Agentic AI Notes
Level: Intermediate Prerequisites: Chapter 1: The LLM as a Reasoning Engine; NLP Notes, Module 7 Time to complete: ~25 minutes
Table of Contents
- Agent Prompts Are Standing Policy
- The Agent System Prompt
- ReAct in Native Tool Calling
- Tree-of-Thought
- Self-Consistency
- Choosing a Pattern
- Summary & Next Steps
1. Agent Prompts Are Standing Policy
The NLP Notes covered prompting for a single response. An agent prompt is a different object: it is evaluated at every step of a run, against a context that changes each time.
The Shift
─────────────────────────────────────────
SINGLE-TURN PROMPT
Read once, against a fixed input.
"Summarise this document."
AGENT SYSTEM PROMPT
Read at every step, against a context that
grows and changes.
"Prefer one broad search over several narrow
ones. Never modify data without confirming."
It is not an instruction. It is POLICY —
a set of rules for decisions not yet encountered.
─────────────────────────────────────────
That reframe changes what belongs in it: not what to do for this task, but how to decide in general.
2. The Agent System Prompt
AGENT_SYSTEM = """You are a support operations agent.
## Objective
Resolve the user's request completely, or explain precisely what blocks you.
## Tools
Use tools to obtain facts. Never state a fact about an account, an order, or a
system that did not come from a tool result in this conversation.
## Decision policy
- Prefer ONE broad query over several narrow ones.
- If two tools could answer, choose the cheaper one (see each tool's cost note).
- If a tool fails twice with the same arguments, try a DIFFERENT approach.
- If you lack information a tool cannot supply, ask the user rather than guessing.
## Stopping
Stop when the request is resolved, when you need user input, or when you have
established the request cannot be fulfilled. State which of the three applies.
## Irreversible actions
Before any tool marked `destructive`, state what you are about to do and why,
then call `request_approval` first. Never chain a destructive action after
another action in the same step.
"""The Five Sections, and Why Each Exists
─────────────────────────────────────────
OBJECTIVE gives the goal-based agent
(Module 1, Ch.3) its completion
test
TOOLS the grounding rule — the single
most effective line against
fabricated facts
DECISION POLICY the utility function (Module 1,
Ch.5) written down, so choices
stop being arbitrary
STOPPING agents fail as often by not
stopping as by stopping early.
Naming the three valid endings
makes "I am blocked" an
acceptable outcome.
IRREVERSIBLE the human gate, stated as policy
ACTIONS rather than enforced only in code
(Module 8 enforces it in code too
— both, always)
─────────────────────────────────────────
The Rule About Rules
─────────────────────────────────────────
Every line in an agent system prompt is read at
EVERY step and billed at every step.
A 2,000-token system prompt on a 15-step run is
30,000 tokens of policy. Keep rules that change
behaviour; delete rules that merely sound
responsible.
─────────────────────────────────────────
3. ReAct in Native Tool Calling
NLP Module 8, Chapter 4 built ReAct with text parsing — Thought:, Action:, Observation:. Native tool calling changed how it is expressed, and the change is worth understanding.
Then and Now
─────────────────────────────────────────
TEXT-PARSED ReAct
Model emits: "Thought: ...\nAction: search[x]"
You regex it. Malformed output breaks the run.
Reasoning is explicit and always present.
NATIVE TOOL CALLING
Model emits a structured tool_call. Cannot be
malformed (Gen AI Notes, Module 2, Ch.3).
But the THOUGHT is now optional — and models
often skip it.
─────────────────────────────────────────
# Recover the "Thought" step explicitly, since native tool calling makes it optional.
REACT_POLICY = """Before every tool call, state in one sentence:
- what you currently know
- what you still need
- why THIS tool answers it
Then make the call."""
def react_step(client, messages, tools):
reply = client.chat.completions.create(
model="gpt-4o", messages=messages, tools=tools,
).choices[0].message
# With the policy above, reply.content carries the reasoning and
# reply.tool_calls carries the action — both, in one response.
if reply.content:
log.info("thought: %s", reply.content) # TRACEABLE (Module 7)
return replyWhy Force the Thought Back In
─────────────────────────────────────────
QUALITY the reason chain-of-thought works
(NLP Notes, Module 7, Ch.2) — the model
conditions its action on its own stated
reasoning rather than jumping.
DEBUGGING a trace of actions tells you WHAT the
agent did. A trace of thoughts tells
you WHY, which is the only way to fix a
bad trajectory.
The cost is a few dozen tokens per step. It is
almost always worth it.
─────────────────────────────────────────
4. Tree-of-Thought
ReAct commits to one path. Tree-of-Thought explores several and selects.
The Structure
─────────────────────────────────────────
problem
/ | \
approach A B C ← generate N candidates
| | |
evaluate each ← score them
| | |
prune ✗ keep ✓ ✗ ← expand only survivors
|
next level
─────────────────────────────────────────
def tree_of_thought(client, problem, breadth=3, depth=2):
frontier = [""]
for _ in range(depth):
candidates = []
for path in frontier:
candidates += propose(client, problem, path, n=breadth) # BRANCH
scored = [(score(client, problem, c), c) for c in candidates] # EVALUATE
frontier = [c for s, c in sorted(scored, reverse=True)[:breadth]] # PRUNE
return frontier[0]The Honest Cost
─────────────────────────────────────────
breadth=3, depth=2 means roughly 3 + 9 proposal
calls plus 12 scoring calls — over 20 LLM calls to
make ONE decision.
Justified when: the decision is expensive to get
wrong, is made once, and has genuinely distinct
alternatives (an architecture choice, a plan for a
long task).
Not justified for: routine tool selection, which
is what most agent steps are. Using ToT everywhere
is the most common way to make an agent 20x more
expensive for no benefit.
─────────────────────────────────────────
5. Self-Consistency
Run the same reasoning several times at non-zero temperature and take the majority answer.
import collections
def self_consistent(client, prompt, n=5, temperature=0.8):
answers = [extract(client.chat.completions.create(
model="gpt-4o", temperature=temperature,
messages=[{"role": "user", "content": prompt}],
).choices[0].message.content) for _ in range(n)]
counts = collections.Counter(answers)
answer, votes = counts.most_common(1)[0]
return answer, votes / n # the AGREEMENT RATE is the useful partThe Underrated Output Is the Agreement Rate
─────────────────────────────────────────
5/5 agreement ──► proceed
3/5 agreement ──► the model is genuinely
uncertain. Escalate, gather
more evidence, or ask a human.
This partially answers Chapter 1's "cannot report
uncertainty" weakness: the model cannot TELL you it
is unsure, but disagreement across samples MEASURES
it.
Use it at decision points that matter, not every
step — it multiplies cost by n.
─────────────────────────────────────────
6. Choosing a Pattern
Decision Guide
─────────────────────────────────────────
Routine step, clear next action
──► ReAct. This is 90% of agent steps.
A branch point where a wrong choice is costly
and hard to undo
──► Tree-of-Thought, once, at that point.
A judgement call with a discrete answer
(classify, approve/deny, pick a route)
──► Self-consistency, and act on the
agreement rate.
A long task needing structure before execution
──► an explicit plan (Module 4), not a
prompting pattern at all.
─────────────────────────────────────────
The Practical Advice
─────────────────────────────────────────
Start with ReAct plus a good system prompt, and
measure. Most agents that "need" ToT actually need
better TOOL DESCRIPTIONS — which is Chapter 3, and
costs nothing per step.
─────────────────────────────────────────
7. Summary & Next Steps
Key Takeaways
- An agent system prompt is standing policy evaluated at every step, so it should encode how to decide in general rather than what to do this time.
- Naming the valid stopping conditions matters as much as naming the goal — agents fail by not stopping at least as often as by stopping early.
- Native tool calling made actions unbreakable but made the reasoning step optional; require it back explicitly for both quality and debuggability.
- Tree-of-Thought and self-consistency multiply cost per decision; reserve them for expensive branch points, and use self-consistency's agreement rate as a proxy for confidence.
Concept Check
- Why is a 2,000-token agent system prompt a different cost proposition from a 2,000-token single-turn prompt?
- What did native tool calling improve about ReAct, and what did it quietly remove?
- An agent must classify a refund request as approve or deny. Which pattern fits, and what would you do with an agreement rate of 3/5?
Next Chapter
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to Agentic AI Index