DBMS

Beyond Relational

Why NoSQL Appeared

A bigger machine. More RAM, faster disks, more

JrCodex·7 min read

Jr Codex DBMS Notes

Level: Intermediate Prerequisites: Module 8, Chapter 3 Time to complete: ~15 minutes

Table of Contents

  1. The Scaling Wall
  2. What Breaks When You Shard
  3. The Other Three Pressures
  4. BASE as an Alternative to ACID
  5. What the Movement Got Right and Wrong
  6. Summary & Next Steps

1. The Scaling Wall

Two Ways to Get Bigger
─────────────────────────────────────────
  VERTICAL (scale up)
    A bigger machine. More RAM, faster disks, more
    cores.
    + nothing about your application changes
    + all relational guarantees intact
    - there is a largest machine, and price rises
      faster than capacity
    - one machine is one failure domain

  HORIZONTAL (scale out)
    More machines.
    + effectively unbounded
    + failure of one is survivable
    - your data is now in several places, and that
      changes everything
─────────────────────────────────────────

Where Relational Databases Sit
─────────────────────────────────────────
  A single PostgreSQL or MySQL instance scales
  vertically very well. Modern hardware handles
  workloads that would have needed a cluster a
  decade ago.

  Read scaling is also easy — add replicas
  (Module 8, Chapter 3).

  WRITE scaling across machines is the hard part,
  because writes must be coordinated, and
  coordination across a network is exactly what is
  expensive.
─────────────────────────────────────────

2. What Breaks When You Shard

Sharding splits one logical table across machines by some key. It is the standard way to scale writes, and it costs you three things.

1. JOINS ACROSS SHARDS
─────────────────────────────────────────
  users on shard A, orders on shard B.

  SELECT ... FROM users JOIN orders ...

  Now requires shipping rows between machines. The
  optimiser's join algorithms (Module 6, Chapter 2)
  assumed local data and cheap page reads. Across a
  network, that assumption is gone.

  Most sharded systems simply DO NOT SUPPORT
  cross-shard joins. You denormalise or you query
  twice and join in application code.

2. TRANSACTIONS ACROSS SHARDS
─────────────────────────────────────────
  Atomicity (Module 1, Chapter 4) across machines
  requires TWO-PHASE COMMIT: a coordinator asks
  every shard to prepare, then tells them all to
  commit.

  It works, and it is slow — several network round
  trips per transaction — and it BLOCKS if the
  coordinator fails mid-protocol, leaving shards
  holding locks with no instruction.

  Many systems refuse multi-shard transactions
  outright rather than offer this.

3. GLOBAL CONSTRAINTS
─────────────────────────────────────────
  UNIQUE on email, when users are spread across 16
  shards.

  Enforcing it means checking all 16 on every
  insert — which is a distributed transaction, with
  all of the above.

  So sharded systems typically only guarantee
  uniqueness WITHIN a shard, and you shard by the
  column you need unique.
─────────────────────────────────────────

The Summary
─────────────────────────────────────────
  Sharding does not make a relational database
  slower. It makes several of its GUARANTEES
  unavailable.

  NoSQL systems were largely built by starting from
  "assume the data is on many machines" and
  designing what remains possible — rather than
  taking a single-machine design and stretching it.
─────────────────────────────────────────

3. The Other Three Pressures

Scale was the loudest reason, but not the only one.

SCHEMA RIGIDITY
─────────────────────────────────────────
  ALTER TABLE on a billion-row table historically
  meant a long lock and a maintenance window.

  Product teams shipping weekly found this
  intolerable, and "schemaless" storage let them
  add fields without a migration.

  Note: this pressure has EASED considerably.
  Modern PostgreSQL and MySQL do most ALTERs
  online, and JSONB columns give schema flexibility
  inside a relational database.

OBJECT-RELATIONAL MISMATCH
─────────────────────────────────────────
  Application objects nest; tables do not
  (Module 1, Chapter 2).

  Loading one user profile might touch six tables.
  Storing it as one document is a single read and
  needs no ORM.

SPECIALISED SHAPES
─────────────────────────────────────────
  Some data genuinely is not tabular:
    - a social graph, queried by traversal
    - time-series, written append-only and read by
      range
    - a full-text corpus

  Relational databases can model all of these. A
  purpose-built system does them substantially
  better.
─────────────────────────────────────────

4. BASE as an Alternative to ACID

The Contrast
─────────────────────────────────────────
  ACID (Module 1, Chapter 4)
    Atomicity, Consistency, Isolation, Durability
    "Correct, or refuse."

  BASE
    Basically Available    responds even during
                           partial failure
    Soft state             state may change without
                           input, as replicas
                           converge
    Eventually consistent  replicas converge GIVEN
                           TIME and no new writes

  "Available, and correct eventually."
─────────────────────────────────────────

What "Eventually" Actually Means
─────────────────────────────────────────
  A write to node A is not immediately visible at
  node B.

  Read from B right after writing to A, and you may
  get the OLD value. Wait, and you will get the new
  one.

  Usually milliseconds. Under a network partition,
  potentially minutes.

  APPLICATIONS MUST BE WRITTEN FOR THIS. "Read your
  own write" stops being automatic — the same
  problem as replication lag (Module 8, Chapter 3),
  but as a designed-in property rather than an
  operational blip.
─────────────────────────────────────────

Where Eventual Consistency Is Fine
─────────────────────────────────────────
  ✓ a social feed a few seconds stale
  ✓ view counts, like counts
  ✓ product recommendations
  ✓ DNS — the internet's most successful eventually
    consistent system (Computer Networks Notes,
    Module 5, Chapter 1)

  ✗ account balances
  ✗ inventory with limited stock
  ✗ "is this seat booked?"
  ✗ anything a user is told succeeded
─────────────────────────────────────────

5. What the Movement Got Right and Wrong

GOT RIGHT
─────────────────────────────────────────
  - One database does not fit every workload
  - Horizontal scale requires giving something up,
    and being explicit about what
  - Some data shapes deserve purpose-built engines
  - Operational simplicity has real value

GOT WRONG
─────────────────────────────────────────
  - "SQL does not scale" — it scales vertically
    far further than most systems ever need
  - "Schemaless means no schema" — the schema moved
    into your application code, where it is
    UNENFORCED and undocumented rather than absent
  - Underestimating how much transactions are worth
    until they were gone
  - Many teams adopted eventual consistency for
    workloads that needed transactions, and
    reimplemented them badly
─────────────────────────────────────────

Where Things Have Settled
─────────────────────────────────────────
  The categories converged.

  Relational databases gained JSON columns,
  logical replication and better online DDL.
  NoSQL systems gained transactions, secondary
  indexes and query languages.
  NewSQL (Chapter 4) offers SQL and ACID over a
  distributed cluster.

  So the modern question is not "SQL or NoSQL". It
  is "which guarantees does this workload need, and
  what is the simplest system that provides them" —
  which is Chapter 4.
─────────────────────────────────────────

6. Summary & Next Steps

Key Takeaways

  • Relational databases scale vertically well and scale reads easily; distributing writes is the hard part, because coordination across a network is what costs.
  • Sharding does not slow a database down — it removes guarantees: cross-shard joins, multi-shard transactions, and global constraints.
  • BASE trades immediate correctness for availability, which is fine for feeds and counters and unacceptable for balances and inventory.
  • "Schemaless" moves the schema into application code where it is unenforced, rather than removing it — the most consequential misunderstanding of the era.

Concept Check

  1. Why is scaling reads straightforward while scaling writes across machines is not?
  2. Name the three guarantees sharding takes away and give a concrete example of each.
  3. Why is "schemaless" a misleading description of a document store?

Next Chapter

→ Chapter 2: The NoSQL Families

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to DBMS Index