Why Databases Exist

Jr Codex DBMS Notes

Level: Beginner Prerequisites: None Time to complete: ~15 minutes

Start With Files
Where the File Approach Breaks
The Five Guarantees
What a DBMS Actually Is
When You Do Not Need One
Summary & Next Steps

1. Start With Files

Every database concept is easier to motivate if you first try to live without one. So: store a shop's orders in a CSV file.

import csv
 
def add_order(customer, item, amount):
    with open("orders.csv", "a", newline="") as f:
        csv.writer(f).writerow([customer, item, amount])
 
def orders_for(customer):
    with open("orders.csv") as f:
        return [row for row in csv.reader(f) if row[0] == customer]

This works. For one user, a few hundred rows, and no concurrency, it is genuinely fine — and Section 5 argues you should keep it that way.

The interesting question is what breaks as this grows.

2. Where the File Approach Breaks

Six Failures, in the Order You Hit Them
─────────────────────────────────────────
  1. SEARCH IS LINEAR
     orders_for() reads EVERY row to find a few.
     At 10 million rows that is seconds per lookup.

  2. NO CONCURRENT WRITES
     Two processes appending at once interleave
     bytes and corrupt the file. There is no lock.

  3. NO ATOMICITY
     "Move $100 from A to B" is two writes. Crash
     between them and the money is gone. The file
     has no notion of a unit of work.

  4. NO INTEGRITY
     Nothing stops amount="banana", a negative
     quantity, or an order for a customer who does
     not exist.

  5. REDUNDANCY AND INCONSISTENCY
     The customer's address is repeated on every
     row. Update it in one place and the file now
     disagrees with itself.

  6. NO SHARED DEFINITION
     Every program that reads the file re-implements
     "column 2 is the amount". Change the format and
     they all break, silently.
─────────────────────────────────────────

The Pattern
─────────────────────────────────────────
  Each failure is a problem you COULD solve
  yourself — an index file, a lock file, a
  write-then-rename, a validation function.

  A DBMS is what you get when someone solves all
  six, correctly, once, so that every application
  does not solve them badly and separately.
─────────────────────────────────────────

3. The Five Guarantees

What you are actually buying, and where each is covered.

1. EFFICIENT ACCESS
─────────────────────────────────────────
  Find rows without scanning everything, via
  indexes. Turns a 10-million-row lookup from
  seconds into microseconds.
  ──► Module 5

2. CORRECTNESS UNDER CONCURRENCY
─────────────────────────────────────────
  Many users reading and writing at once, each
  seeing a consistent view, without corrupting
  each other.
  ──► Module 7

3. SURVIVING FAILURE
─────────────────────────────────────────
  Pull the power mid-write and the database comes
  back consistent — every committed change present,
  every uncommitted one gone.
  ──► Module 8

4. ENFORCED INTEGRITY
─────────────────────────────────────────
  Rules declared ONCE in the schema — types, keys,
  required fields, valid ranges, relationships —
  and enforced for every writer, forever.
  ──► Modules 2 and 4

5. A DECLARATIVE INTERFACE
─────────────────────────────────────────
  You say WHAT you want; the system decides HOW to
  get it. This is what SQL is, and it is why the
  same query keeps working when the data grows a
  thousandfold and the optimiser quietly switches
  strategy.
  ──► Modules 3 and 6
─────────────────────────────────────────

The fifth is the one people underrate. In the CSV version, you wrote the loop. Change the access pattern and you rewrite the code. In SQL, you describe the result and the database re-plans on your behalf.

4. What a DBMS Actually Is

Worth separating three words that get used interchangeably.

Term	Means
Database	The data itself — the actual stored rows
DBMS	The software managing it: PostgreSQL, MySQL, SQLite, Oracle
Database system	Both together, plus the applications using them

The Same Task, With a DBMS
─────────────────────────────────────────
  You declare the rules ONCE:

    CREATE TABLE orders (
      id       INTEGER PRIMARY KEY,
      customer TEXT    NOT NULL REFERENCES customers(id),
      item     TEXT    NOT NULL,
      amount   NUMERIC NOT NULL CHECK (amount > 0)
    );

  And every one of the six failures is now handled:
    linear search  ──► the index on id
    concurrency    ──► the lock manager
    atomicity      ──► transactions
    integrity      ──► NOT NULL, CHECK, REFERENCES
    redundancy     ──► customer is a REFERENCE, not
                       a copy
    definition     ──► the schema IS the shared
                       definition
─────────────────────────────────────────

import sqlite3
 
conn = sqlite3.connect("shop.db")
conn.execute("PRAGMA foreign_keys = ON")        # SQLite needs this switched on explicitly
conn.executescript("""
    CREATE TABLE IF NOT EXISTS customers (
        id   INTEGER PRIMARY KEY,
        name TEXT NOT NULL
    );
    CREATE TABLE IF NOT EXISTS orders (
        id       INTEGER PRIMARY KEY,
        customer INTEGER NOT NULL REFERENCES customers(id),
        item     TEXT    NOT NULL,
        amount   NUMERIC NOT NULL CHECK (amount > 0)
    );
""")
 
conn.execute("INSERT INTO customers (id, name) VALUES (1, 'Asha')")
conn.execute("INSERT INTO orders (customer, item, amount) VALUES (1, 'Keyboard', 45.00)")
conn.commit()
 
# The rules now defend themselves:
try:
    conn.execute("INSERT INTO orders (customer, item, amount) VALUES (99, 'Mouse', 20)")
except sqlite3.IntegrityError as e:
    print("rejected:", e)          # FOREIGN KEY constraint failed — customer 99 does not exist

5. When You Do Not Need One

A database is not free. It is a dependency, an operational burden, and a thing to learn.

A File Is Genuinely Better When
─────────────────────────────────────────
  - one writer, and no concurrency
  - the data fits in memory and is read whole
  - the format matters more than the queries
    (config files, CSV exports, logs)
  - it must be human-readable and diffable
  - it is a cache you can rebuild
─────────────────────────────────────────

The Honest Middle Ground
─────────────────────────────────────────
  SQLite is a full relational database in a single
  file, with no server, no configuration and no
  process to run.

  It gives you transactions, indexes, constraints
  and SQL for roughly the effort of opening a file
  — which is why this curriculum uses it for the
  first six modules, and why "we do not need a
  database yet" is usually a false economy.
─────────────────────────────────────────

6. Summary & Next Steps

Key Takeaways

Storing data in files fails in six predictable ways: linear search, unsafe concurrent writes, no atomicity, no integrity enforcement, redundancy, and no shared definition of the format.
A DBMS is those six problems solved once and correctly, so every application does not solve them separately and badly.
The five guarantees are efficient access, correctness under concurrency, survival of failure, enforced integrity, and a declarative interface — and each maps to a later module.
The declarative interface is the most underrated: you describe the result, and the database re-plans how to get it as the data grows.

Concept Check

A script transfers money between two accounts by writing two lines to a file. What specifically goes wrong if the process is killed between the writes, and which guarantee addresses it?
Why does storing a customer's address on every order row cause a correctness problem, not just a storage one?
Give a concrete case where a plain file is the better choice, and say what makes it so.

Next Chapter

→ Chapter 2: Data Models and the Relational Choice

Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to DBMS Index