Advanced Prompting: Chain-of-Thought & Few-Shot Techniques

Jr Codex NLP & LLM Notes

Level: Intermediate–Advanced Prerequisites: Chapter 1: Prompting Fundamentals Time to complete: ~25 minutes

Beyond Basic Instructions
Few-Shot Prompting, Revisited
Chain-of-Thought Prompting
Why Chain-of-Thought Works
Zero-Shot Chain-of-Thought
Self-Consistency
Least-to-Most Prompting
Summary & Next Steps

1. Beyond Basic Instructions

Chapter 1 covered the fundamentals of clear, well-structured prompts. This chapter covers techniques that meaningfully improve performance on harder tasks — especially multi-step reasoning, math, and logic — beyond what a straightforward instruction achieves.

2. Few-Shot Prompting, Revisited

Module 5, Chapter 3 introduced few-shot prompting as an in-context learning technique. This section covers practical guidance for constructing effective few-shot examples specifically.

Guidance for Choosing Few-Shot Examples
─────────────────────────────────────────
  DIVERSITY:      cover the RANGE of input variations the
                    model will actually see — not just similar
                    examples
  CONSISTENT format:   use the identical structure/formatting
                          across ALL examples, so the model can
                          reliably infer the PATTERN (Module 5,
                          Ch.3, Section 3)
  ORDER can matter:       some research shows examples CLOSER
                             to the actual query (at the END of
                             the prompt) can have OUTSIZED
                             influence — worth testing different
                             orderings if results seem sensitive
                             to it
  CORRECT examples ONLY:     an incorrect or LOW-quality example
                                 can actively mislead the model
                                 (Section 1's in-context learning
                                 mechanism means the model may
                                 genuinely LEARN from a bad
                                 example, within that single
                                 prompt)
─────────────────────────────────────────

3. Chain-of-Thought Prompting

Chain-of-Thought (CoT) prompting asks the model to explicitly show its reasoning steps, one at a time, before giving a final answer — dramatically improving accuracy on tasks requiring multi-step reasoning.

# WITHOUT chain-of-thought
standard_prompt = """Q: Roger has 5 tennis balls. He buys 2 more cans of tennis
balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A:"""
# A model might answer INCORRECTLY, especially for MORE complex versions of this problem
 
# WITH chain-of-thought
cot_prompt = """Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.
Each can has 3 tennis balls. How many tennis balls does he have now?
A: Let's think step by step. Roger starts with 5 tennis balls. He buys 2 cans,
each containing 3 balls, so that's 2 x 3 = 6 more balls. In total, he has
5 + 6 = 11 tennis balls.
 
Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and half
of the golf balls are blue. How many blue golf balls are there?
A: Let's think step by step."""

CoT as a FEW-SHOT Technique
─────────────────────────────────────────
  Notice the prompt STRUCTURE: it's actually Section 2's
  few-shot approach, but each EXAMPLE demonstrates the
  REASONING PROCESS explicitly, not just the final answer —
  teaching the model, WITHIN this single prompt, to show its
  work on the NEW problem too.
─────────────────────────────────────────

4. Why Chain-of-Thought Works

The Leading Explanation
─────────────────────────────────────────
  A decoder-only model (Module 5, Chapter 1) generates its
  final answer TOKEN BY TOKEN, using EVERYTHING generated so
  far (including its OWN previous tokens) as context for the
  next prediction (DL Notes, Module 6, Ch.2's self-attention).

  By generating INTERMEDIATE reasoning steps FIRST, the model
  effectively "shows its work" — and each subsequent token's
  prediction can DIRECTLY attend to and build on those
  intermediate steps, rather than needing to solve the ENTIRE
  multi-step problem "in one shot," compressed into a SINGLE
  token prediction.

  This connects DIRECTLY to Module 5, Chapter 3, Section 5's
  emergent abilities discussion — CoT's benefit is itself
  documented as an EMERGENT property, providing LITTLE benefit
  for smaller models but SUBSTANTIAL benefit for sufficiently
  large ones.
─────────────────────────────────────────

5. Zero-Shot Chain-of-Thought

Remarkably, simply appending the phrase "Let's think step by step" to a prompt — with no worked examples at all — produces much of CoT's benefit, for sufficiently capable models.

zero_shot_cot_prompt = """Q: A store had 120 apples. They sold 45% of them in the
morning and 30 more in the afternoon. How many apples are left?
A: Let's think step by step."""

Why This "Magic Phrase" Works
─────────────────────────────────────────
  The pretraining corpus (Module 3, Ch.3, Section 6) almost
  certainly contains COUNTLESS examples of step-by-step
  reasoning FOLLOWING this exact (or very similar) phrase —
  the model has learned, from raw text, that this phrase
  PREDICTS a certain kind of methodical, structured
  continuation, and simply completing that learned PATTERN
  produces genuinely useful reasoning steps.
─────────────────────────────────────────

This is a striking, concrete illustration of Module 5, Chapter 3, Section 3's explanation for why in-context learning works at all.

6. Self-Consistency

Self-consistency improves on chain-of-thought further: generate multiple independent reasoning chains (using Module 5, Chapter 1's temperature sampling for diversity), then take the majority vote among their final answers.

from collections import Counter
 
def self_consistency_prompt(model, tokenizer, prompt, num_samples=5):
    answers = []
    for _ in range(num_samples):
        inputs = tokenizer(prompt, return_tensors="pt")
        output = model.generate(**inputs, max_new_tokens=150, do_sample=True, temperature=0.7)
        response = tokenizer.decode(output[0], skip_special_tokens=True)
        final_answer = extract_final_answer(response)      # a task-specific parsing function
        answers.append(final_answer)
 
    most_common_answer = Counter(answers).most_common(1)[0][0]
    return most_common_answer, answers

Why Majority Voting Improves Accuracy
─────────────────────────────────────────
  Different SAMPLED reasoning chains (Section 3) may take
  slightly different PATHS, and occasionally make DIFFERENT
  mistakes along the way — but if the model's UNDERLYING
  reasoning ability is genuinely sound, the CORRECT answer is
  more likely to appear CONSISTENTLY across multiple
  independent attempts than any SPECIFIC incorrect answer
  (which would need to occur the SAME way, by chance, multiple
  times to "win" the vote).
─────────────────────────────────────────

This comes at a direct COST — generating multiple full reasoning chains multiplies inference compute and cost, a genuine tradeoff to weigh against the accuracy gain, directly connecting to Module 9's production cost concerns.

7. Least-to-Most Prompting

For problems that are naturally DECOMPOSABLE into sequential sub-problems, least-to-most prompting explicitly breaks the task into stages: first prompt the model to DECOMPOSE the problem into sub-questions, THEN solve each sub-question in sequence, using previous sub-answers as context for later ones.

decomposition_prompt = """Break down this problem into simpler sub-questions:
"If a train travels 60 mph for 2.5 hours, then 45 mph for 1.5 hours,
what is the total distance traveled?"
 
Sub-questions:
1. How far does the train travel in the first leg (60 mph for 2.5 hours)?
2. How far does the train travel in the second leg (45 mph for 1.5 hours)?
3. What is the sum of both legs' distances?"""
 
# Each sub-question is then solved IN SEQUENCE, with EARLIER answers
# provided as CONTEXT for later sub-questions — directly echoing DL
# Notes, Module 5, Chapter 4's sequence-to-sequence, step-by-step framing

8. Summary & Next Steps

Key Takeaways

Chain-of-thought prompting asks the model to show reasoning steps explicitly before a final answer, substantially improving accuracy on multi-step reasoning tasks.
Chain-of-thought's benefit is itself an emergent ability — it helps large models substantially more than small ones, connecting directly to the emergence discussion in Module 5.
Zero-shot chain-of-thought ("let's think step by step") works because pretraining data contains countless examples of methodical reasoning following similar phrases.
Self-consistency generates multiple independent reasoning chains and takes a majority vote, improving accuracy at the direct cost of increased inference compute.

Concept Check

Why does generating intermediate reasoning steps before a final answer improve accuracy, given how a decoder-only model generates text token by token?
Why does simply appending "let's think step by step" improve performance even without any worked examples?
What's the tradeoff self-consistency introduces in exchange for its accuracy improvement?

Next Chapter

→ Chapter 3: Structured Output & Function Calling

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to NLP & LLM Index