Computer Networks

The Transport Layer

Flow and Congestion Control

Solved with the receive WINDOW, advertised in

JrCodex·9 min read

Jr Codex Computer Networks Notes

Level: Intermediate–Advanced Prerequisites: Chapter 3: TCP Connections and Reliability Time to complete: ~25 minutes

Table of Contents

  1. Two Different Problems
  2. Flow Control and the Sliding Window
  3. Window Scaling
  4. Congestion Collapse
  5. Slow Start and AIMD
  6. Modern Congestion Control
  7. Summary & Next Steps

1. Two Different Problems

The Distinction
─────────────────────────────────────────
  FLOW CONTROL
    Protects the RECEIVER.
    "Do not send faster than I can process."
    Solved with the receive WINDOW, advertised in
    every ACK.

  CONGESTION CONTROL
    Protects the NETWORK.
    "Do not send faster than the path can carry."
    Solved with a CONGESTION WINDOW the sender
    computes for itself.

  A sender is limited by the SMALLER of the two:

    effective window = min(rwnd, cwnd)
─────────────────────────────────────────

Why Both Are Needed
─────────────────────────────────────────
  A fast server sending to a phone: the PHONE is
  the bottleneck. Flow control.

  A fast server sending to a fast client over a
  congested backbone: the NETWORK is the
  bottleneck, and the receiver has plenty of room.
  Congestion control.

  Neither mechanism can detect the other's problem.
─────────────────────────────────────────

2. Flow Control and the Sliding Window

The Mechanism
─────────────────────────────────────────
  The receiver advertises a WINDOW in every ACK:
  how many more bytes it can buffer.

  The sender may have at most that many
  unacknowledged bytes in flight.

     sent+ACKed │ sent, not ACKed │ may send │ later
    ────────────┼─────────────────┼──────────┼──────
                └──── the window ────────────┘
                      slides right as ACKs arrive
─────────────────────────────────────────

class SlidingWindow:
    def __init__(self, rwnd):
        self.rwnd = rwnd
        self.send_base = 0            # oldest unacknowledged byte
        self.next_seq  = 0            # next byte to send
 
    def can_send(self, nbytes):
        in_flight = self.next_seq - self.send_base
        return in_flight + nbytes <= self.rwnd        # never exceed the window
 
    def on_ack(self, ack_num, new_rwnd):
        self.send_base = max(self.send_base, ack_num)  # the window SLIDES
        self.rwnd = new_rwnd                           # and may RESIZE
THE ZERO WINDOW, AND THE DEADLOCK IT WOULD CAUSE
─────────────────────────────────────────
  A receiver whose buffer is full advertises
  window = 0. The sender stops.

  When the application drains the buffer, the
  receiver sends a window update.

  IF THAT UPDATE IS LOST, both sides wait forever —
  the sender for permission, the receiver for data.

  FIX: the PERSIST TIMER. The sender periodically
  transmits a one-byte WINDOW PROBE, forcing the
  receiver to re-advertise its window.

  A small mechanism that exists purely to prevent
  one specific deadlock.
─────────────────────────────────────────

3. Window Scaling

The 16-Bit Problem
─────────────────────────────────────────
  The window field is 16 bits, so the maximum
  advertised window is 65,535 bytes.

  Recall the BANDWIDTH-DELAY PRODUCT (Module 1,
  Chapter 5): a 1 Gbps link with 80ms RTT has a BDP
  of 10 MB.

  A 64 KB window on that path achieves:
    65,535 bytes / 0.08 s ≈ 6.5 Mbps

  0.65% of the link. The sender transmits 64 KB,
  then WAITS for an ACK with the pipe nearly empty.
─────────────────────────────────────────

def max_throughput_mbps(window_bytes, rtt_ms):
    return (window_bytes * 8) / (rtt_ms / 1000) / 1e6
 
print(f"{max_throughput_mbps(65_535,      80):>8.1f} Mbps")   # 6.6  — unscaled
print(f"{max_throughput_mbps(4_194_304,   80):>8.1f} Mbps")   # 419.4 — scale factor 6
print(f"{max_throughput_mbps(16_777_216,  80):>8.1f} Mbps")   # 1677.7 — saturates 1 Gbps
THE FIX: WINDOW SCALE OPTION
─────────────────────────────────────────
  A TCP option, negotiated in the HANDSHAKE ONLY,
  giving a left-shift factor of 0-14. The effective
  window becomes up to 1 GB.

  Because it is negotiated only in the SYN, a
  middlebox that strips or mangles TCP options
  silently disables it — and the connection is
  capped at 64 KB.

  SYMPTOM: a fast link performs terribly on long
  paths and fine on short ones. It is a classic and
  hard-to-find problem.
─────────────────────────────────────────

4. Congestion Collapse

What Happened in 1986
─────────────────────────────────────────
  The NSFNET backbone's throughput fell from
  32 kbps to 40 BPS — a factor of 1,000.

  THE MECHANISM:
    1. Routers become congested and drop packets
    2. Senders time out and RETRANSMIT
    3. Retransmissions ADD LOAD to an already
       overloaded network
    4. More congestion ──► more loss ──► more
       retransmission
    5. Nearly all traffic is retransmissions of
       packets that will also be dropped

  The network is fully utilised and delivering
  almost nothing. This is CONGESTION COLLAPSE.
─────────────────────────────────────────

THE INSIGHT THAT FIXED IT
─────────────────────────────────────────
  PACKET LOSS IS A SIGNAL OF CONGESTION.

  So a sender that detects loss should SLOW DOWN,
  not merely retransmit.

  Van Jacobson's congestion control, added in 1988,
  made every TCP sender back off when it saw loss —
  turning a positive feedback loop into a negative
  one.

  It is a VOLUNTARY, END-HOST mechanism. No router
  enforces it. The internet works because nearly
  every endpoint chooses to be well-behaved — which
  is why the UDP warning in Chapter 2 matters.
─────────────────────────────────────────

5. Slow Start and AIMD

The Two Phases
─────────────────────────────────────────
  SLOW START — exponential growth
    cwnd starts at ~10 segments.
    Every ACK increases cwnd by one segment, so it
    DOUBLES every RTT.
    "Slow" is historical: it is slow to START, and
    then very fast.
    Continues until cwnd reaches ssthresh, or loss
    occurs.

  CONGESTION AVOIDANCE — linear growth
    cwnd increases by ONE segment per RTT.
    Probing gently for more capacity.
─────────────────────────────────────────

AIMD — Additive Increase, Multiplicative Decrease
─────────────────────────────────────────
  INCREASE  +1 segment per RTT   (cautious)
  DECREASE  ×0.5 on loss         (drastic)

  The asymmetry is deliberate: probe slowly,
  retreat quickly.

  AIMD provably CONVERGES TO FAIRNESS. Two flows
  sharing a link, whatever their starting rates,
  approach an equal share — because the
  multiplicative decrease reduces the larger flow
  by more in absolute terms.
─────────────────────────────────────────

class Reno:
    """TCP Reno's congestion window, the classic AIMD implementation."""
 
    def __init__(self, mss=1460):
        self.mss = mss
        self.cwnd = 10 * mss                    # initial window
        self.ssthresh = 64 * 1024
        self.state = "slow_start"
 
    def on_ack(self, acked_bytes):
        if self.cwnd < self.ssthresh:
            self.state = "slow_start"
            self.cwnd += self.mss               # EXPONENTIAL: +1 MSS per ACK
        else:
            self.state = "congestion_avoidance"
            self.cwnd += self.mss * self.mss / self.cwnd    # LINEAR: +1 MSS per RTT
 
    def on_triple_duplicate_ack(self):
        """Mild signal: packets are still flowing. FAST RECOVERY."""
        self.ssthresh = max(self.cwnd // 2, 2 * self.mss)
        self.cwnd = self.ssthresh               # halve, do NOT restart
 
    def on_timeout(self):
        """Severe signal: nothing is getting through."""
        self.ssthresh = max(self.cwnd // 2, 2 * self.mss)
        self.cwnd = self.mss                    # back to ONE segment
        self.state = "slow_start"
The Two Loss Signals Are Treated Differently
─────────────────────────────────────────
  TRIPLE DUPLICATE ACK
    ACKs are still arriving, so packets are still
    getting through. One was lost. MILD.
    ──► halve the window and continue.

  TIMEOUT
    Nothing came back at all. The path may be
    severely congested or broken. SEVERE.
    ──► reset to one segment and slow-start again.

  Distinguishing these is TCP Reno's main
  improvement over its predecessor, and it roughly
  doubled throughput on lossy paths.
─────────────────────────────────────────

6. Modern Congestion Control

CUBIC — the default on Linux and most systems
─────────────────────────────────────────
  Reno's linear growth is far too slow on
  high-BDP paths. Recovering a 10 MB window at one
  segment per RTT takes thousands of round trips.

  CUBIC grows the window as a CUBIC FUNCTION of
  time since the last loss:

    - fast growth immediately after a reduction
    - FLATTENS near the previous maximum, probing
      cautiously around the known-good point
    - accelerates again if no loss occurs, to find
      newly available capacity

  It is also RTT-INDEPENDENT, so a long-distance
  flow is not starved by a short-distance one
  sharing the link — which Reno does badly.

BBR — a different signal entirely
─────────────────────────────────────────
  Every algorithm above treats LOSS as the
  congestion signal. That assumption has two
  problems:

  BUFFERBLOAT — routers with huge buffers absorb
  excess packets instead of dropping them. Loss
  happens only after enormous queuing delay has
  built up, so loss-based TCP fills those buffers
  and adds hundreds of milliseconds of latency
  before it slows down.

  WIRELESS LOSS — a corrupted frame on wifi is not
  congestion, but loss-based TCP halves its window
  anyway (Module 1, Chapter 5's 1% loss finding).

  BBR instead MODELS the path: it estimates the
  bottleneck BANDWIDTH and the minimum RTT, and
  sends at exactly that rate.

  RESULT: high throughput with LOW queuing delay,
  and no over-reaction to random loss.
─────────────────────────────────────────

# Inspect and change the algorithm (Linux).
sysctl net.ipv4.tcp_congestion_control          # cubic
sysctl net.ipv4.tcp_available_congestion_control
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr
 
# The window scale and SACK settings from Section 3.
sysctl net.ipv4.tcp_window_scaling net.ipv4.tcp_sack
BUFFERBLOAT, and Why You Have Felt It
─────────────────────────────────────────
  A large download makes video calls stutter and
  web pages crawl on the SAME connection — even
  though bandwidth remains.

  The download's TCP flow has filled the router's
  buffer, so every other packet queues behind
  megabytes of it. Latency rises from 20ms to
  several hundred.

  FIXES: BBR on the sender, and ACTIVE QUEUE
  MANAGEMENT on the router — CoDel or fq_codel,
  which drop packets early rather than buffering
  endlessly. If your home router supports
  fq_codel, enabling it is the single best latency
  improvement available.
─────────────────────────────────────────

7. Summary & Next Steps

Key Takeaways

  • Flow control protects the receiver via an advertised window; congestion control protects the network via a window the sender computes, and the sender is limited by whichever is smaller.
  • The 16-bit window field caps throughput at 6.5 Mbps on an 80ms path unless window scaling is negotiated — and a middlebox stripping TCP options silently reimposes that cap.
  • Congestion collapse happened because retransmission added load; the fix was treating loss as a congestion signal and backing off, voluntarily, at every end host.
  • AIMD converges to fairness, CUBIC fixes Reno's slow recovery on high-BDP paths, and BBR abandons loss as the signal entirely to avoid bufferbloat and wireless false positives.

Concept Check

  1. Why is a sender limited by min(rwnd, cwnd) rather than either alone?
  2. Explain why a triple duplicate ACK and a timeout are treated as different severities.
  3. Why does a large download make an unrelated video call stutter, and what fixes it?

Next Chapter

→ Chapter 5: TCP in the Real World

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to Computer Networks Index