Agentic AI

Planning And Self Correction

Reflection & Self-Correction

Module 2, Chapter 1 listed self-assessment among the reasoning core's weaknesses. Reflection is built on exactly that weakness, so the first question is whether

JrCodex·8 min read

Jr Codex Agentic AI Notes

Level: Advanced Prerequisites: Chapter 2: Planning Patterns Time to complete: ~25 minutes

Table of Contents

  1. The Self-Assessment Problem
  2. Critique-and-Revise
  3. Grounding the Critique
  4. Reflexion — Learning Within a Run
  5. Verification Beats Reflection
  6. When Reflection Is Worth Its Cost
  7. Summary & Next Steps

1. The Self-Assessment Problem

Module 2, Chapter 1 listed self-assessment among the reasoning core's weaknesses. Reflection is built on exactly that weakness, so the first question is whether it can work at all.

The Empty Version
─────────────────────────────────────────
  Agent: [produces an answer]
  You:   "Is that correct?"
  Agent: "Yes, that is correct."

  This measures nothing. The model has no
  independent access to the truth — it is sampling
  again from the same distribution that produced the
  answer, now with the answer in context making
  agreement the most likely continuation.
─────────────────────────────────────────

Why Reflection Nonetheless Works Sometimes
─────────────────────────────────────────
  GENERATING and CHECKING are different tasks.

  Producing a correct SQL query requires getting
  every clause right at once. Noticing that a query
  lacks a WHERE clause requires checking one thing.

  Reflection helps when the check is genuinely
  EASIER than the generation, and helps not at all
  when it is the same difficulty.
─────────────────────────────────────────

That distinction predicts everything in this chapter. Reflection on "did I use the right tool" works. Reflection on "is this fact true" does not.

2. Critique-and-Revise

The basic loop: produce, critique against explicit criteria, revise.

from pydantic import BaseModel
from typing import Literal
 
class Critique(BaseModel):
    issues: list[str]                       # concrete problems, each actionable
    verdict: Literal["accept", "revise"]
 
CRITIC_PROMPT = """Evaluate this output against the criteria. Be specific and concrete.
 
CRITERIA:
{criteria}
 
TASK: {task}
OUTPUT: {output}
 
List only issues you can point to in the output. Do not invent problems.
If it meets every criterion, return an empty issue list and verdict "accept"."""
 
def critique_and_revise(client, task, criteria, max_rounds=2):
    output = generate(client, task)
 
    for _ in range(max_rounds):
        c = client.chat.completions.parse(
            model="gpt-4o",
            messages=[{"role": "user", "content": CRITIC_PROMPT.format(
                criteria=criteria, task=task, output=output)}],
            response_format=Critique, temperature=0,
        ).choices[0].message.parsed
 
        if c.verdict == "accept":
            return output, c
        output = revise(client, task, output, c.issues)      # FIX the named issues
 
    return output, c
Three Design Requirements
─────────────────────────────────────────
  EXPLICIT CRITERIA
    "Is this good?" produces vague, unusable
    critiques. A written checklist produces
    actionable ones. This is the same finding as the
    evaluation rubrics in the Gen AI Notes,
    Module 5, Chapter 3.

  A SEPARATE CALL
    Critiquing in the same call that generated the
    output biases heavily toward "accept". Fresh
    context, ideally a different model.

  "DO NOT INVENT PROBLEMS"
    Without it, a critic asked to find issues WILL
    find issues, every round, forever. This line is
    what makes `accept` reachable.
─────────────────────────────────────────

3. Grounding the Critique

An ungrounded critic is guessing. A grounded one is checking.

def grounded_critique(client, task, output, tools):
    """Give the critic EVIDENCE, not just the output."""
    evidence = {
        "schema": tools.get_schema(),                    # what actually exists
        "test_result": tools.run_tests(output),          # did it run?
        "lint": tools.lint(output),                      # mechanical checks
        "sources": tools.cited_sources(output),          # do the citations resolve?
    }
    return client.chat.completions.parse(
        model="gpt-4o",
        messages=[{"role": "user", "content":
            f"TASK: {task}\nOUTPUT: {output}\n\nEVIDENCE:\n{evidence}\n\n"
            f"Identify issues the EVIDENCE demonstrates. Cite which evidence "
            f"shows each issue. Do not raise issues the evidence does not support."}],
        response_format=Critique, temperature=0,
    ).choices[0].message.parsed
The Hierarchy of Critique Quality
─────────────────────────────────────────
  WEAKEST   "Is this good?"
            Ungrounded, no criteria. Near-useless.

  BETTER    "Does this meet criteria X, Y, Z?"
            Grounded in a rubric.

  STRONG    "The test suite reports 2 failures and
             the schema has no `orders.region`
             column. What is wrong?"
            Grounded in FACTS from the environment.

  The strongest form barely uses the model's
  judgement at all — it uses the model to INTERPRET
  evidence that something else produced.
─────────────────────────────────────────

4. Reflexion — Learning Within a Run

Reflexion extends critique with memory: after a failed attempt, write a lesson and keep it in context for the next attempt.

def reflexion(agent, task, verify, max_attempts=3):
    """Retry with accumulated lessons rather than blindly."""
    reflections = []
 
    for attempt in range(max_attempts):
        result = agent.run(task, reflections=reflections)     # prior lessons in context
        check = verify(result)                                # EXTERNAL check — Section 5
 
        if check.passed:
            return result, reflections
 
        reflection = agent.client.chat.completions.create(
            model="gpt-4o", temperature=0,
            messages=[{"role": "user", "content":
                f"TASK: {task}\nATTEMPT: {result}\nIT FAILED BECAUSE: {check.reason}\n\n"
                f"In two sentences, state what to do DIFFERENTLY next time. "
                f"Be specific about the approach, not the outcome."}],
        ).choices[0].message.content
 
        reflections.append(f"Attempt {attempt + 1} failed: {check.reason}\n"
                           f"Next time: {reflection}")
 
    return result, reflections
What Makes It Work
─────────────────────────────────────────
  A NON-EMPTY FAILURE SIGNAL
    `verify` must say WHY, not just "no". "Test
    test_refund_rounding failed: expected 10.00 got
    10.004" is actionable. "Incorrect" is not.

  "APPROACH, NOT OUTCOME"
    Without this, reflections read "I should have
    got the right answer" — true, useless. Forcing
    a statement about METHOD produces something the
    next attempt can act on.

  IN-CONTEXT ONLY
    Reflexion is Module 1, Chapter 3's
    "learning within a run". Nothing persists unless
    you also write it to procedural memory
    (Module 3, Chapter 3) — which is usually worth
    doing.
─────────────────────────────────────────

5. Verification Beats Reflection

The most important point in this chapter.

The Hierarchy
─────────────────────────────────────────
  1. DETERMINISTIC VERIFICATION
       compiler, test suite, schema validator,
       type checker, a query that returns rows
       ──► TRUE or FALSE. Use this wherever it
           exists.

  2. TOOL-GROUNDED CHECKING
       re-query the source and compare
       ──► evidence, not opinion

  3. LLM CRITIQUE with explicit criteria
       ──► useful, fallible

  4. LLM SELF-ASSESSMENT
       "did I do well?"
       ──► approximately worthless

  Always climb as high as your task allows.
─────────────────────────────────────────

def verify(result, task) -> Check:
    """Prefer mechanical checks; fall back to a model only when nothing else applies."""
    if task.kind == "code":
        return run_tests(result)                       # level 1 — unambiguous
    if task.kind == "sql":
        return validate_against_schema(result)         # level 1
    if task.kind == "extraction":
        return schema_check(result, task.schema)       # level 1
    if task.kind == "research":
        return check_citations_resolve(result)         # level 2 — every claim sourced?
    return llm_critique(result, task.criteria)         # level 3 — last resort
The Design Consequence
─────────────────────────────────────────
  When you can choose the task's OUTPUT FORMAT,
  choose one that is mechanically checkable.

  "Write a summary"          ──► level 3 or 4
  "Write a summary where     ──► level 2: citations
   every claim cites a            can be resolved
   retrieved source id"           automatically

  A small change in what you ask for moves the whole
  task up two levels of verification quality. This
  is the same argument the Gen AI Notes made for
  structured output.
─────────────────────────────────────────

6. When Reflection Is Worth Its Cost

Reflection multiplies cost and latency by the number of rounds. It is not free and not always positive.

WORTH IT
─────────────────────────────────────────
  - A cheap, real verifier exists (tests, schema)
  - The output is expensive to get wrong
  - The task has objective criteria
  - Failure is silent otherwise

NOT WORTH IT
─────────────────────────────────────────
  - Subjective output with no criteria
  - Simple tasks the model gets right first time
  - Latency-sensitive paths
  - No verification signal — you are just asking
    the model to agree with itself

The Diminishing Returns
─────────────────────────────────────────
  Round 1  catches most real issues
  Round 2  catches a few more
  Round 3+ mostly churn — the critic starts
           inventing issues to justify its
           existence, and quality can DECLINE

  Two rounds is the standard cap. Measure whether
  the second one helps on YOUR task before keeping
  it.
─────────────────────────────────────────

7. Summary & Next Steps

Key Takeaways

  • Asking a model whether its own answer is correct measures nothing; reflection works only where checking is genuinely easier than generating.
  • A useful critique needs explicit criteria, a separate call, and an instruction not to invent problems — otherwise accept is never reachable.
  • Reflexion needs a failure signal that says why, and reflections must state a different approach rather than a desired outcome.
  • Deterministic verification beats every form of self-critique, so prefer output formats that are mechanically checkable — that choice moves the task up two levels of verification quality.

Concept Check

  1. Why does reflection help with "did I choose the right tool" but not with "is this fact true"?
  2. What happens to a critique loop that omits "do not invent problems," and why?
  3. You are asked to have an agent produce a research summary. What one change to the required output would move verification from level 3 to level 2?

Next Chapter

→ Chapter 4: When Planning Fails

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to Agentic AI Index