Concurrency Basics

Jr Codex Python Notes

Level: Advanced Prerequisites: Chapter 5 Time to complete: ~30 minutes

Table of Contents

1. Concurrency vs Parallelism
2. CPU-Bound vs I/O-Bound Work
3. The Global Interpreter Lock (GIL)
4. Threading
5. Multiprocessing
6. asyncio Fundamentals
7. Choosing the Right Tool
8. Summary & Next Steps

1. Concurrency vs Parallelism

These terms get used interchangeably in casual conversation, but they mean different things:

Concurrency                          Parallelism
─────────────────────────────         ─────────────────────────────
  Managing MULTIPLE tasks,              Actually RUNNING multiple tasks
  making progress on each,               at the EXACT SAME instant
  by interleaving/switching              (requires multiple CPU cores)
  between them (may be on ONE core)
─────────────────────────────         ─────────────────────────────
  Like a chef juggling three            Like three chefs, each cooking
  dishes on one stove, switching         their own dish, simultaneously,
  attention between them                 on three separate stoves

Python's tools map to different points on this spectrum — this chapter covers all three main approaches: threading (concurrency, limited by the GIL), multiprocessing (true parallelism), and asyncio (concurrency, single-threaded).

2. CPU-Bound vs I/O-Bound Work

This distinction determines which concurrency tool actually helps:

# CPU-bound: the bottleneck is the CPU doing computation
def cpu_bound_task():
    total = 0
    for i in range(100_000_000):
        total += i ** 2
    return total
 
# I/O-bound: the bottleneck is WAITING (network, disk, database) — CPU is idle
import time
def io_bound_task():
    time.sleep(2)          # simulates waiting for a network response
    return "data received"

3. The Global Interpreter Lock (GIL)

Python's standard implementation (CPython) has a Global Interpreter Lock — only one thread can execute Python bytecode at any given instant, even on a multi-core machine.

Without GIL (ideal, other languages)     With Python's GIL
─────────────────────────────           ─────────────────────────────
  Core 1: Thread A running                Core 1: Thread A running
  Core 2: Thread B running                Core 2: Thread B WAITING for the GIL
  Core 3: Thread C running                Core 3: Thread C WAITING for the GIL
  (true parallel execution)                (only ONE thread executes Python at a time)
─────────────────────────────           ─────────────────────────────

Why does this matter? It means Python threads do not speed up CPU-bound work — they can't run Python code on multiple cores simultaneously. But threads do help I/O-bound work, because a thread waiting on the network releases the GIL, letting another thread run in the meantime.

4. Threading

Best for I/O-bound tasks — multiple things waiting on external resources at once.

import threading
import time
 
def download_file(name, duration):
    print(f"Starting download: {name}")
    time.sleep(duration)          # simulates waiting on a network response
    print(f"Finished download: {name}")
 
start = time.perf_counter()
 
threads = []
files = [("file1.zip", 2), ("file2.zip", 2), ("file3.zip", 2)]
 
for name, duration in files:
    thread = threading.Thread(target=download_file, args=(name, duration))
    threads.append(thread)
    thread.start()          # starts running CONCURRENTLY, doesn't block here
 
for thread in threads:
    thread.join()              # wait for each thread to finish before continuing
 
print(f"Total time: {time.perf_counter() - start:.2f} seconds")
# Total time: ~2.0 seconds — NOT 6 seconds! The three "downloads" overlapped.

Sequential (no threading)              Threaded
─────────────────────────────         ─────────────────────────────
  file1 (2s) → file2 (2s) → file3 (2s)   file1, file2, file3 all "waiting" AT ONCE
  Total: 6 seconds                       Total: ~2 seconds (limited by the SLOWEST one)
─────────────────────────────         ─────────────────────────────

A Common Threading Problem: Race Conditions

import threading
 
counter = 0
 
def increment():
    global counter
    for _ in range(100_000):
        counter += 1          # NOT atomic! Multiple threads can interfere with each other here
 
threads = [threading.Thread(target=increment) for _ in range(5)]
for t in threads:
    t.start()
for t in threads:
    t.join()
 
print(counter)      # Expected: 500,000 — but often prints something LESS, unpredictably!

This is a race condition — multiple threads reading/modifying shared data at the same time, stepping on each other. The fix is a Lock:

import threading
 
counter = 0
lock = threading.Lock()
 
def increment():
    global counter
    for _ in range(100_000):
        with lock:              # only ONE thread can hold the lock at a time
            counter += 1
 
threads = [threading.Thread(target=increment) for _ in range(5)]
for t in threads:
    t.start()
for t in threads:
    t.join()
 
print(counter)      # 500,000 — correct, every time, thanks to the lock

5. Multiprocessing

Best for CPU-bound tasks — sidesteps the GIL entirely by running each task in its own separate process (its own Python interpreter, its own memory), enabling true parallel execution across CPU cores.

import multiprocessing
import time
 
def cpu_heavy_task(n):
    total = 0
    for i in range(n):
        total += i ** 2
    return total
 
if __name__ == "__main__":          # REQUIRED on Windows for multiprocessing!
    start = time.perf_counter()
 
    with multiprocessing.Pool(processes=4) as pool:
        results = pool.map(cpu_heavy_task, [10_000_000] * 4)
 
    print(f"Results: {results}")
    print(f"Total time: {time.perf_counter() - start:.2f} seconds")
    # Roughly 4x faster than running the four calls sequentially, on a 4+ core machine

Threading (GIL-limited)                Multiprocessing (true parallelism)
─────────────────────────────         ─────────────────────────────
  All threads share ONE process,        Each process has its OWN interpreter,
  ONE GIL — only helps I/O waiting        OWN memory — genuinely runs on
                                           separate CPU cores at once
─────────────────────────────         ─────────────────────────────

Trade-off: processes don't share memory directly (unlike threads) — passing data between them has real overhead (serialization), and starting a process is heavier than starting a thread. Use multiprocessing when the CPU-bound work genuinely justifies that cost.

6. asyncio Fundamentals

asyncio provides concurrency within a single thread, using async/await to cooperatively switch between tasks whenever one is waiting (I/O). No threads, no GIL contention, no race conditions on shared data — just one thread juggling many waiting tasks efficiently.

import asyncio
 
async def download_file(name, duration):
    print(f"Starting download: {name}")
    await asyncio.sleep(duration)      # non-blocking "wait" — lets OTHER tasks run meanwhile
    print(f"Finished download: {name}")
    return f"{name} complete"
 
async def main():
    results = await asyncio.gather(
        download_file("file1.zip", 2),
        download_file("file2.zip", 2),
        download_file("file3.zip", 2),
    )
    print(results)
 
asyncio.run(main())
# All three "downloads" run concurrently — total time ~2 seconds, not 6

Key asyncio Vocabulary
─────────────────────────────────────────
  async def   → defines a COROUTINE function (doesn't run immediately when called)
  await        → pauses THIS coroutine until the awaited thing completes,
                   letting other coroutines run in the meantime
  asyncio.run() → the entry point that actually starts the event loop
  gather()       → run multiple coroutines concurrently, wait for all to finish
─────────────────────────────────────────

Critical rule: await only yields control at points where it's waiting on something (I/O, asyncio.sleep, a network call). A CPU-bound loop inside an async def function will still block everything else — asyncio does not help CPU-bound work at all; that's multiprocessing's job.

# This does NOT get any concurrency benefit — no await points during the heavy computation
async def bad_cpu_task():
    total = 0
    for i in range(100_000_000):     # blocks the ENTIRE event loop while this runs
        total += i ** 2
    return total

7. Choosing the Right Tool

Decision Guide
─────────────────────────────────────────
  Is the work CPU-bound (heavy computation)?
     → multiprocessing

  Is the work I/O-bound (waiting on network/disk/APIs)?
     → Many simultaneous connections, modern codebase?  → asyncio
     → Working with existing blocking/threaded libraries? → threading
─────────────────────────────────────────

In practice for this curriculum: most of the ML/DL/data work in later modules relies on libraries (NumPy, PyTorch) that release the GIL internally during heavy number-crunching, or use multiprocessing under the hood (e.g., PyTorch's DataLoader workers) — you'll rarely write raw threading/multiprocessing code yourself, but understanding why these tools exist explains a lot of behavior you'll encounter in those libraries.

8. Summary & Next Steps

Key Takeaways

• Concurrency (interleaving progress on multiple tasks) is not the same as parallelism (simultaneous execution) — Python's GIL means threads give you the former, not the latter, for CPU-bound work.
• Use multiprocessing for CPU-bound work (true parallelism across cores); use threading or asyncio for I/O-bound work (overlapping waiting time).
• Shared mutable state across threads causes race conditions — protect it with a Lock.
• asyncio's async/await gives single-threaded concurrency for I/O-bound code, but provides zero benefit for CPU-bound loops — those still block the entire event loop.

Module 5 Complete — Next Module

You now have the advanced language toolkit: generators, decorators, context managers, type hints, functional utilities, dataclasses, testing, logging, and concurrency fundamentals. Module 6 is the bridge into data-focused Python — NumPy and pandas — directly connecting everything you've learned to the Machine Learning module that follows this entire Python curriculum.

Concept Check

1. Why doesn't threading speed up a CPU-bound task in standard Python?
2. What's a race condition, and what tool fixes it in threaded code?
3. Why does an async def function with a heavy CPU-bound loop (and no await inside it) fail to provide any concurrency benefit?

Next Chapter

→ Module 6: Bridge to Data & ML

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index