Rag And Agents
Agents & Tool Use
Module 7, Chapter 3, Section 6 previewed this exact connection: an agent is fundamentally a loop of function calls, where the model decides — after each result
JrCodex·6 min read
Jr Codex NLP & LLM Notes
Level: Advanced Prerequisites: Chapter 3: Advanced RAG Techniques Time to complete: ~25 minutes
Table of Contents
- From a Single Tool Call to an Agent
- The Agent Loop
- The ReAct Pattern
- Implementing a Simple Agent
- Giving an Agent Multiple Tools
- Why Agents Are Genuinely Risky
- Summary & Next Steps
1. From a Single Tool Call to an Agent
Module 7, Chapter 3, Section 6 previewed this exact connection: an agent is fundamentally a loop of function calls, where the model decides — after each result — whether the task is complete or another action is needed. This directly extends the AI Notes' agent framework: perceive, decide, act, repeat.
2. The Agent Loop
The General Structure
─────────────────────────────────────────
1. The LLM receives the current STATE (user request +
history of previous actions/results)
2. The LLM decides: is the task COMPLETE, or is another
ACTION needed?
3. If an action is needed: the LLM specifies WHICH tool to
call and WITH WHAT arguments (Module 7, Ch.3's function
calling)
4. The APPLICATION executes the tool, and the RESULT is fed
back into the conversation history
5. REPEAT from step 1, until the LLM decides the task is
complete
─────────────────────────────────────────
Mapping This Onto the AI Notes' Agent Framework
─────────────────────────────────────────
Perceive: the current conversation history + latest
tool result
Decide: the LLM's reasoning about what to do NEXT
(Module 7, Ch.2's chain-of-thought is
often used explicitly HERE)
Act: executing the chosen tool
─────────────────────────────────────────
3. The ReAct Pattern
ReAct (Reasoning + Acting) is the dominant pattern for structuring an agent's internal process: at each step, the model explicitly generates a Thought (reasoning about what to do), an Action (which tool to call), and receives an Observation (the tool's result) — directly combining Module 7, Chapter 2's chain-of-thought with tool use.
A ReAct Trace, Illustrated
─────────────────────────────────────────
User: "What's the weather in the city where the 2024 Olympics
were held?"
Thought: I need to find out which city hosted the 2024
Olympics first, then check its weather.
Action: search("2024 Olympics host city")
Observation: "The 2024 Summer Olympics were held in Paris,
France."
Thought: Now I need the current weather in Paris.
Action: get_weather("Paris")
Observation: {"temperature": 18, "condition": "cloudy"}
Thought: I now have enough information to answer.
Final Answer: The weather in Paris, host of the 2024 Olympics,
is currently 18°C and cloudy.
─────────────────────────────────────────
Why Explicit "Thought" Steps Help
─────────────────────────────────────────
Directly echoing Module 7, Chapter 2, Section 4's
explanation of WHY chain-of-thought works — generating
explicit reasoning BEFORE choosing an action lets the model
"show its work" and build on its OWN prior reasoning via
self-attention (DL Notes, Module 6, Ch.2), generally
producing MORE reliable tool selection and argument
construction than jumping directly to an action.
─────────────────────────────────────────
4. Implementing a Simple Agent
def run_agent(client, user_query, tools, tool_functions, max_steps=5):
messages = [{"role": "user", "content": user_query}]
for step in range(max_steps):
response = client.chat.completions.create(
model="gpt-4", messages=messages, tools=tools,
)
message = response.choices[0].message
if not message.tool_calls: # the model decided NO further action is needed (Step 2)
return message.content # the FINAL answer
messages.append(message)
for tool_call in message.tool_calls: # Step 3-4 — execute EACH requested tool
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
result = tool_functions[function_name](**function_args) # ACTUALLY execute it
messages.append({
"role": "tool", "tool_call_id": tool_call.id, "content": json.dumps(result),
})
# LOOP BACK to step 1 — the LLM sees the new result and decides the NEXT step
return "Max steps reached without a final answer." # a SAFETY LIMIT (Section 6)The max_steps limit is not optional — without it, a confused or looping agent could call tools indefinitely, an important safety consideration covered further in Section 6.
5. Giving an Agent Multiple Tools
tools = [
{"type": "function", "function": {
"name": "search_web", "description": "Search the web for current information",
"parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]},
}},
{"type": "function", "function": {
"name": "get_weather", "description": "Get current weather for a location",
"parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]},
}},
{"type": "function", "function": {
"name": "calculate", "description": "Evaluate a mathematical expression",
"parameters": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]},
}},
]
tool_functions = {
"search_web": lambda query: search_web_implementation(query),
"get_weather": lambda location: get_weather(location),
"calculate": lambda expression: eval(expression), # a REAL implementation would use a SAFE
# math parser, NOT raw eval() — see Section 6
}Clear, Distinct Tool Descriptions Matter
─────────────────────────────────────────
The model chooses WHICH tool to call based ENTIRELY on the
natural-language "description" field (Module 7, Ch.3, Sec.4)
— AMBIGUOUS or OVERLAPPING tool descriptions lead directly
to the model choosing the WRONG tool, or being unable to
decide between similar options confidently.
─────────────────────────────────────────
6. Why Agents Are Genuinely Risky
Real Risks, Not Just Hypothetical Ones
─────────────────────────────────────────
UNSAFE tool implementations: the `eval()` placeholder
above is a genuine SECURITY
risk — a malicious or
confused query could
inject ARBITRARY code;
REAL agent tools need
careful, SANDBOXED
implementations
Infinite or excessive loops: without max_steps
(Section 4), a
confused agent could
call tools
indefinitely, incurring
unbounded COST (each
LLM call and tool
execution costs money/
compute)
Irreversible ACTIONS: an agent with
access to tools
that send
emails, make
purchases, or
modify data
can take
REAL-WORLD,
HARD-TO-UNDO
actions based
on a
MISUNDERSTOOD
request —
directly
echoing this
session's OWN
safety
guidelines
around
confirming
risky actions
before taking
them
Prompt injection: if RETRIEVED
content
(Chapter
2-3) or
tool
RESULTS
contain
malicious
instructions,
an agent
might
follow
THEM
instead
of the
original
user's
intent
─────────────────────────────────────────
Practical Mitigations
─────────────────────────────────────────
- Require HUMAN CONFIRMATION before executing IRREVERSIBLE
actions (sending, deleting, purchasing)
- Use SANDBOXED, restricted tool implementations (never raw
eval(), restrict file system/network access)
- Set REASONABLE max_steps and cost/rate limits
- Treat ALL retrieved/tool-result content as potentially
UNTRUSTED input, not as trusted instructions
─────────────────────────────────────────
7. Summary & Next Steps
Key Takeaways
- An agent is fundamentally a loop of function calls, where the model decides after each tool result whether the task is complete or further action is needed.
- The ReAct pattern structures this loop with explicit Thought/Action/Observation steps, combining chain-of-thought reasoning with tool use.
- A max_steps safety limit is essential to prevent a confused agent from calling tools indefinitely, incurring unbounded cost.
- Agents carry genuine risks — unsafe tool implementations, irreversible actions, and prompt injection from untrusted retrieved content — requiring careful mitigation, not just capability-focused design.
Concept Check
- How does the ReAct pattern's explicit "Thought" step improve on jumping directly to an action?
- Why is a max_steps limit a genuine safety necessity, not just an implementation detail?
- What is prompt injection in the context of agents, and why does it matter for RAG-connected agents specifically?
Next Chapter
→ Chapter 5: Multi-Agent Systems
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to NLP & LLM Index