Computer Networks

The Transport Layer

Ports, Sockets and Multiplexing

That machine is running a web server, an SSH

JrCodex·8 min read

Jr Codex Computer Networks Notes

Level: Intermediate Prerequisites: Module 3, Chapter 5: NAT, DHCP and ICMP Time to complete: ~20 minutes

Table of Contents

  1. The Gap IP Leaves
  2. Port Numbers
  3. The Connection 4-Tuple
  4. Sockets
  5. A Server and a Client
  6. Connection States and TIME_WAIT
  7. Summary & Next Steps

1. The Gap IP Leaves

What IP Delivers
─────────────────────────────────────────
  A packet arrives at 203.0.113.5.

  That machine is running a web server, an SSH
  daemon, a database and a mail server.

  WHICH ONE gets the packet?

  IP has no answer — its address identifies a
  MACHINE, not a PROGRAM.
─────────────────────────────────────────

The Transport Layer's Two Jobs
─────────────────────────────────────────
  1. MULTIPLEXING / DEMULTIPLEXING
     Deliver to the right PROCESS, using PORTS.
     Both UDP and TCP do this.

  2. RELIABILITY
     Turn IP's lossy, unordered delivery into an
     ordered, reliable byte stream.
     TCP does this. UDP deliberately does not.

  Job 1 is this chapter. Job 2 is Chapters 3 and 4.
─────────────────────────────────────────

2. Port Numbers

The Ranges
─────────────────────────────────────────
  16 bits, so 0-65535.

  0     - 1023   WELL KNOWN
                 Assigned by IANA. On Unix, binding
                 these requires root — a security
                 measure so an unprivileged user
                 cannot impersonate a system
                 service.

  1024  - 49151  REGISTERED
                 Assigned to specific applications
                 on request. PostgreSQL 5432,
                 MySQL 3306.

  49152 - 65535  DYNAMIC / EPHEMERAL
                 Assigned automatically to CLIENTS
                 for the duration of a connection.
─────────────────────────────────────────

Worth Memorising
─────────────────────────────────────────
   20/21  FTP data / control
     22   SSH
     25   SMTP
     53   DNS (both UDP and TCP)
     67/68 DHCP server / client
     80   HTTP
    123   NTP
    143   IMAP
    443   HTTPS
    3306  MySQL
    5432  PostgreSQL
    6379  Redis
    8080  HTTP alternate
─────────────────────────────────────────

Client Ports Are Not Special
─────────────────────────────────────────
  A server LISTENS on a known port so clients can
  find it.

  A client's port is chosen at random from the
  ephemeral range, because nobody needs to find the
  client — the server replies to whatever port the
  request came from.

  This asymmetry is the whole reason well-known
  ports exist: discovery in one direction only.
─────────────────────────────────────────

3. The Connection 4-Tuple

What Identifies a Connection
─────────────────────────────────────────
  (source IP, source port,
   destination IP, destination port)

  Plus the protocol, technically making it a
  5-tuple.

  Two connections may share three of the four and
  still be distinct.
─────────────────────────────────────────

Why a Server Handles Thousands on One Port
─────────────────────────────────────────
  A web server listens on port 443. Ten thousand
  clients connect. All ten thousand connections
  have the same destination IP and destination
  port.

  They are distinguished by the SOURCE side:

    (203.0.113.9,  51234, 10.0.0.1, 443)
    (203.0.113.9,  51235, 10.0.0.1, 443)  ← same
                                             client
    (198.51.100.7, 51234, 10.0.0.1, 443)  ← same
                                             port,
                                             other
                                             client

  All three are different tuples, so all three are
  different connections.

  This is why "one port can only handle one
  connection" is wrong, and it is a common
  misunderstanding.
─────────────────────────────────────────

# Demultiplexing, modelled: the OS looks up the tuple to find the socket.
connections = {}          # 4-tuple -> socket
 
def demultiplex(packet):
    key = (packet.src_ip, packet.src_port, packet.dst_ip, packet.dst_port)
    if key in connections:
        return connections[key]                        # an established connection
 
    listening = (packet.dst_ip, packet.dst_port)
    if listening in listeners and packet.syn:
        sock = accept_new(packet)                      # a NEW connection
        connections[key] = sock
        return sock
 
    return None            # nothing listening ──► TCP sends RST, UDP sends ICMP

4. Sockets

The Abstraction
─────────────────────────────────────────
  A SOCKET is the operating system's handle for one
  endpoint of a communication.

  It is a FILE DESCRIPTOR — so read(), write() and
  close() work on it, which is why network code
  looks like file code on Unix.

  Two main types:
    SOCK_STREAM   TCP — a reliable byte stream
    SOCK_DGRAM    UDP — independent messages
─────────────────────────────────────────

THE STREAM vs MESSAGE DISTINCTION
─────────────────────────────────────────
  This causes more real bugs than anything else in
  socket programming.

  UDP PRESERVES MESSAGE BOUNDARIES.
    Send 100 bytes, then 50 ──► the receiver gets
    one 100-byte read and one 50-byte read.

  TCP DOES NOT.
    Send 100 bytes, then 50 ──► the receiver may
    get 150 in one read, or 30 then 120, or any
    other split.

  TCP is a BYTE STREAM. It guarantees the bytes
  arrive in order; it guarantees nothing about how
  they are grouped into reads.

  So EVERY TCP protocol must define its own
  framing — a length prefix, a delimiter, or a
  fixed size. This is Module 2, Chapter 2's framing
  problem reappearing one layer up.
─────────────────────────────────────────

5. A Server and a Client

import socket
 
def server(host="0.0.0.0", port=9000):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)   # see Section 6
    s.bind((host, port))                     # claim the port
    s.listen(128)                            # the ACCEPT QUEUE depth
    print(f"listening on {host}:{port}")
 
    while True:
        conn, addr = s.accept()              # blocks; returns a NEW socket per client
        with conn:
            print(f"connection from {addr}")
            while True:
                data = conn.recv(4096)
                if not data:                 # empty ──► the peer closed
                    break
                conn.sendall(data.upper())
def client(host, port, message: bytes):
    with socket.create_connection((host, port), timeout=5) as s:
        s.sendall(message)
 
        # TCP is a STREAM: one sendall may need several recv calls.
        received = bytearray()
        while len(received) < len(message):
            chunk = s.recv(4096)
            if not chunk:
                raise ConnectionError("peer closed early")
            received.extend(chunk)
        return bytes(received)
# Length-prefix framing — the standard fix for TCP's lack of boundaries.
import struct
 
def send_message(sock, payload: bytes):
    sock.sendall(struct.pack("!I", len(payload)) + payload)    # 4-byte length, then data
 
def recv_exactly(sock, n: int) -> bytes:
    buf = bytearray()
    while len(buf) < n:
        chunk = sock.recv(n - len(buf))
        if not chunk:
            raise ConnectionError("peer closed mid-message")
        buf.extend(chunk)
    return bytes(buf)
 
def recv_message(sock) -> bytes:
    (length,) = struct.unpack("!I", recv_exactly(sock, 4))     # read the length first
    return recv_exactly(sock, length)                          # then exactly that many
Note the listen() Backlog
─────────────────────────────────────────
  listen(128) sets how many completed connections
  may wait for accept().

  If the application is slow to accept and the
  queue fills, new connections are refused or
  dropped — and the symptom is intermittent
  connection failures under load, with the server
  looking healthy.
─────────────────────────────────────────

6. Connection States and TIME_WAIT

The States You Will See
─────────────────────────────────────────
  LISTEN        a server waiting for connections
  SYN_SENT      a client has sent SYN
  SYN_RECEIVED  a server has received SYN, sent
                SYN-ACK
  ESTABLISHED   the connection is open
  FIN_WAIT_1/2  closing, initiated by this side
  CLOSE_WAIT    the peer closed; THIS side has not
  TIME_WAIT     closed, waiting before releasing
                the tuple
  CLOSED        gone
─────────────────────────────────────────

ss -tan | awk '{print $1}' | sort | uniq -c | sort -rn
#   4821 ESTAB
#   1203 TIME-WAIT
#     87 CLOSE-WAIT      ← this one is a bug signal
TWO STATES THAT DIAGNOSE REAL BUGS
─────────────────────────────────────────
  MANY TIME_WAIT
    NORMAL on a busy client or proxy. The side that
    closes FIRST holds TIME_WAIT for 2×MSL
    (typically 60s) so late-arriving packets from
    the old connection cannot be mistaken for a new
    one reusing the same tuple.
    It is a correctness feature, not a leak.
    Mitigate with connection reuse (keep-alive),
    not by disabling it.

  MANY CLOSE_WAIT
    ALWAYS AN APPLICATION BUG. The peer closed, the
    OS told your application, and your application
    never called close().
    The file descriptor leaks. Eventually you run
    out and the process cannot open anything.
    Fix the code — no kernel setting helps.
─────────────────────────────────────────

SO_REUSEADDR
─────────────────────────────────────────
  Without it, restarting a server fails with
  "address already in use" because old connections
  sit in TIME_WAIT holding the port.

  SO_REUSEADDR lets the new listener bind anyway.
  It is safe and standard — which is why it appears
  in essentially every server example, including
  the one above.
─────────────────────────────────────────

7. Summary & Next Steps

Key Takeaways

  • IP addresses a machine; ports address a program, and the transport layer's first job is demultiplexing to the right process.
  • A connection is identified by the 4-tuple, which is why one listening port serves thousands of simultaneous connections.
  • TCP is a byte stream with no message boundaries, so every TCP protocol must define its own framing — usually a length prefix.
  • Many TIME_WAIT sockets is normal and protective; many CLOSE_WAIT sockets is always an application that forgot to close, and leaks file descriptors.

Concept Check

  1. Ten thousand clients connect to one server on port 443. How does the OS tell the connections apart?
  2. Your code calls send twice with 100 and 50 bytes, and the receiver's first recv returns 150 bytes. Is this a bug? Explain.
  3. A server accumulates CLOSE_WAIT sockets over days until it stops accepting connections. What is wrong and where is the fix?

Next Chapter

→ Chapter 2: UDP

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to Computer Networks Index