Prompt Engineering
Prompting Fundamentals
Every technique through Module 6 involved changing a model's weights. Prompt engineering changes nothing about the model — it's the skill of crafting input text
Jr Codex NLP & LLM Notes
Level: Intermediate Prerequisites: Module 6: Fine-Tuning & Aligning LLMs Time to complete: ~20 minutes
Table of Contents
- A Genuinely Different Skill
- Anatomy of a Good Prompt
- System Prompts vs User Prompts
- Being Specific and Unambiguous
- Providing Context
- Common Prompting Mistakes
- Iterating on Prompts Systematically
- Summary & Next Steps
1. A Genuinely Different Skill
Every technique through Module 6 involved changing a model's weights. Prompt engineering changes nothing about the model — it's the skill of crafting input text that reliably elicits the desired behavior from an already-trained, unchanged model, exploiting Module 5, Chapter 3's in-context learning directly.
Why This Is Worth a Dedicated Module
─────────────────────────────────────────
Prompting is FREE (no training cost, DL Notes Module 9's
GPU/compute concerns don't apply), INSTANT to iterate on
(no training time), and often SURPRISINGLY effective —
per Module 6, Chapter 4, Section 2's decision order, it
should be the FIRST thing tried before reaching for
fine-tuning.
─────────────────────────────────────────
2. Anatomy of a Good Prompt
Common Components of an Effective Prompt
─────────────────────────────────────────
Role/persona: "You are an expert technical writer..."
— sets the STYLE and expertise level of
the response
Task description: a CLEAR, specific statement of what's
being asked
Context/input: the actual DATA or content the
task should be performed on
Format specification: HOW the output should be
structured (bullet points,
JSON, a specific length)
Constraints: things to AVOID, or
boundaries to respect
─────────────────────────────────────────
prompt = """You are an expert technical writer who explains complex topics simply.
Task: Explain the concept of recursion in programming.
Format: Provide a one-paragraph explanation, followed by a simple
code example in Python.
Constraints: Avoid jargon. Assume the reader has never programmed before."""3. System Prompts vs User Prompts
Most modern LLM APIs distinguish between a system prompt (set once, establishing persistent behavior/persona) and user prompts (the actual, per-turn conversation).
from openai import OpenAI # illustrative — any major LLM provider's API follows a similar pattern
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content": "You are a helpful assistant that always responds in formal English and never uses contractions."},
{"role": "user", "content": "Can you help me plan a birthday party?"},
],
)Why Separate System and User Prompts
─────────────────────────────────────────
The system prompt establishes CONSISTENT behavior across an
ENTIRE conversation (tone, persona, output format
preferences) WITHOUT needing to repeat it in every single
user message — directly analogous to Module 6, Chapter 1's
instruction-tuning DATA format, but specified at INFERENCE
time instead of TRAINING time.
─────────────────────────────────────────
4. Being Specific and Unambiguous
Directly echoing Module 1, Chapter 1's discussion of language's inherent ambiguity — a vague prompt gives the model room to guess, and it may guess differently than intended.
Vague vs Specific Prompts
─────────────────────────────────────────
VAGUE: "Write something about dogs."
→ could produce a poem, a factual article, a
story, ANYTHING remotely dog-related
SPECIFIC: "Write a 100-word factual paragraph about
how dogs' sense of smell works, suitable
for a children's science magazine."
→ dramatically narrows the SPACE of plausible,
reasonable outputs
─────────────────────────────────────────
# A concrete before/after example
vague_prompt = "Summarize this."
specific_prompt = """Summarize the following article in exactly 3 bullet points,
each under 15 words, focusing on the article's main findings (not background
context or methodology)."""5. Providing Context
An LLM has no knowledge beyond its training data and whatever is provided within the prompt itself — directly foreshadowing Module 8's RAG chapter, which automates the process of finding and providing relevant context.
prompt_without_context = "What does our return policy say about opened electronics?"
# The model has NO idea what "our" refers to — it can only guess or hallucinate
prompt_with_context = """Based on the following return policy, answer the question.
Policy: "Electronics may be returned within 30 days if unopened. Opened
electronics are eligible for store credit only, within 14 days of purchase."
Question: What does our return policy say about opened electronics?"""6. Common Prompting Mistakes
Frequent Beginner Mistakes
─────────────────────────────────────────
Assuming the model KNOWS an LLM only knows what's in
something not actually its training data or the
provided: CURRENT prompt — it
cannot access external
files, databases, or
"common knowledge about
YOUR specific
situation" unless
EXPLICITLY provided
(Section 5)
Overly LONG, unfocused burying the actual TASK
prompts: within excessive,
irrelevant detail makes
it harder for the
model to identify
what's actually being
asked
Not specifying OUTPUT the model will GUESS a
FORMAT: reasonable format if
none is specified —
often NOT the format
a downstream system
actually needs to
parse (Module 7,
Ch.3)
Expecting PERFECT accuracy LLMs can still
on every response: produce
confidently
wrong answers
("hallucinations")
— verification
(Module 9)
remains
essential for
high-stakes
use cases
─────────────────────────────────────────
7. Iterating on Prompts Systematically
A Practical Iteration Loop
─────────────────────────────────────────
1. Write an INITIAL prompt, based on Section 2's anatomy
2. Test against SEVERAL representative inputs (not just one)
3. Identify SPECIFIC failure patterns (wrong format? missing
constraint? misunderstood task?)
4. Revise the prompt to address THAT specific failure
5. Re-test — INCLUDING previously-passing cases, to catch
any REGRESSION introduced by the revision
6. Repeat until performance is consistently acceptable
─────────────────────────────────────────
This directly mirrors the ML Notes' iterative model development workflow — prompting is, in a real sense, a lightweight, training-free analog of that same iterative refinement discipline. Module 9 covers systematic, automated prompt evaluation in depth.
8. Summary & Next Steps
Key Takeaways
- Prompt engineering changes no model weights — it crafts input text that reliably elicits desired behavior from an already-trained model, exploiting in-context learning directly.
- A well-structured prompt typically includes role/persona, task description, context/input, format specification, and constraints.
- System prompts establish persistent behavior across a conversation, while user prompts carry the per-turn content.
- Specificity and explicitly provided context are essential — an LLM only knows its training data and what's directly included in the current prompt.
Concept Check
- Why should prompt engineering generally be tried before fine-tuning, per Module 6's decision order?
- Why does a vague prompt like "write something about dogs" produce unpredictable results?
- Why can't an LLM answer a question about "our return policy" without that policy being included in the prompt?
Next Chapter
→ Chapter 2: Advanced Prompting — Chain-of-Thought & Few-Shot Techniques
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to NLP & LLM Index