Before You Change a Live System, Write Down What It Does Today

In the previous post I argued for writing a spec before handing work to an AI agent. That works neatly for a new feature. But most real work isn't a new feature — it's a change to a system that has been in production for years, where nobody has a complete picture of what it actually does.

In that situation, the first spec you need isn't for the change. It's a baseline spec: a written description of how the system behaves today. Every later spec is then written as a difference from it.

I recently wrote one for a large production platform before a planned rebuild. This post covers the structure I used and the lessons from it.

The code is the spec — and nobody can read it

An older system does have a spec. It's the code. The problem is that it's spread across hundreds of files, scheduled jobs and configuration, and some of it contradicts the documentation.

So questions like these take hours:

• "How long does a sign-in code last?"
• "What happens if a payment confirmation arrives twice?"
• "When does a student lose access after their plan expires?"

And if you ask an AI agent to change the system without that picture, it will happily build on assumptions that aren't true.

A baseline spec turns the code back into something people can read, review and argue with.

Rule one: describe what is, not what should be

The most important line in the document is at the top:

This is a baseline: it records what the production platform does today,
not what it should do. Nothing here is a request to build anything.

This rule is harder to follow than it sounds. While reading code you constantly spot things you'd do differently, and the temptation is to write the spec the way the system ought to behave. Don't. A baseline that mixes current behaviour with wishes is useless as a reference — you can no longer tell which statements are true.

The structure

I used the same shape as a feature spec, with a few additions for the as-is case.

User stories with priorities

Each major journey gets a short story, a priority, a reason for the priority, and an independent test — a way to check the whole story end to end:

### Student takes a practice test (Priority: P1)

A student builds a practice test by choosing a course and subject,
the number of questions and a time limit, answers it and submits.

Why this priority: practice tests are a core learning loop.

Independent test: create a 10-question test, submit it within the
time limit, and confirm it appears in the student's history.

Acceptance scenarios:
1. Given a request for fewer than 10 questions, then the test is refused.
2. Given a test whose time limit ran out more than 5 minutes ago,
   when it is submitted, then the submission is refused.

Priorities matter even in a baseline. They tell a reviewer where to spend their verification time, and they tell a rebuild which journeys must not break.

Numbered requirements with real numbers

Every rule gets an ID and uses MUST language:

- FR-001: Students MUST sign in with a one-time SMS code that
  expires after 120 seconds.
- FR-002: Code requests MUST be limited to 5 per number per
  5-minute window.

The IDs are what make the baseline useful later. A future spec can say "this changes FR-002" and a reviewer knows exactly what's moving.

And use the actual production values — the ones in code and configuration — not the ones in old tickets. Half the value of a baseline is replacing "I think it's about two minutes" with a number someone checked.

Edge cases, especially timing

Production systems are full of behaviour that is eventually correct. Background jobs, caches and scheduled runs mean a user can see stale state for minutes or hours. These rarely appear in any documentation, and they generate a large share of support tickets, so give them their own section:

- Access expiry is applied by a job every 6 hours, so access can
  last up to 6 hours past its end date.
- Structural changes can take hours to appear in the cached
  course tree.

Writing these down turns "it's random" bug reports into documented, explainable behaviour.

Rule two: bugs are not requirements

Here's the part that makes a baseline different from a feature spec. When you document existing behaviour, you will find behaviour that is clearly unintended. You can't write it as a requirement — then the spec would demand the bug. And you can't silently leave it out — then the spec would be wrong about what the system does.

So it goes in a separate section, Known Deviations, each with an ID and a severity:

| ID    | Sev | Area     | Current behaviour |
|-------|-----|----------|-------------------|
| KD-01 | C   | Commerce | An admin export includes data it shouldn't. |
| KD-02 | M   | Exams    | Users see three different "results ready" times. |
| KD-03 | M   | Content  | An error message states the wrong limit. |

Each deviation needs a product decision: fix it (it becomes a planned task) or accept it (it becomes a requirement). Some things that look like bugs turn out to be deliberate — a loophole that's really a promotion, a limit that's intentionally loose. The baseline's job is to surface them, not to judge them.

One practical note: a list like this, for a live product, is sensitive. It belongs in the internal repository with restricted access — never in a public document or a blog post.

Rule three: record where the docs are wrong

Reading the code against the existing documentation will turn up places where they disagree: a doc that says a feature "isn't built yet" when it already is, or a described bug that can't actually be reached. List these explicitly rather than picking a side quietly. Whoever owns the documentation decides which one is right.

Success criteria for the spec itself

A baseline is only useful if people trust it, so it needs its own definition of done:

- SC-001: Before approval, a domain owner has confirmed every
  requirement against production behaviour, or corrected it.
- SC-002: Within 2 weeks of approval, every critical deviation
  has a decision (fix or accept) and an owner.
- SC-003: Every later feature spec cites the baseline IDs it
  adds, changes or removes.
- SC-004: A new team member can answer "what happens when X?"
  for any area in under 15 minutes without reading code.

The first one matters most. A baseline produced by reading code is a hypothesis about production. Until someone who knows the domain confirms it, it's a draft.

Where AI fits

Reading an entire codebase and writing down every rule, limit and scheduled job is exactly the kind of work AI agents are good at — tireless, thorough, and fast across hundreds of files. I used an agent to do the reading and drafting, with a few instructions that made a big difference:

• State only what the code does. Cite production configuration for every number.
• Never write unintended behaviour as a requirement. Put it under Known Deviations instead.
• Flag conflicts with existing docs instead of resolving them.
• Keep implementation details out. Describe behaviour a user or operator would notice, not class names.

The human part is the part that matters: confirming the requirements, deciding each deviation, and owning the result.

After the baseline

Once it's approved, the baseline changes how every later piece of work starts:

1. A new feature spec describes its change relative to the baseline and cites the IDs it touches.
2. Reviewers check the change against the baseline instead of reverse-engineering the old behaviour each time.
3. When a change ships, the baseline is updated so it stays true.

For a rebuild, it's also the checklist: the new system is done when it satisfies every requirement — and every deviation has been either fixed or deliberately kept.

Takeaways

• For an existing system, write a baseline spec before speccing changes.
• Describe what the system does today, not what it should do.
• Give every requirement an ID and the real production numbers.
• Document timing and eventual-consistency behaviour as edge cases.
• Put unintended behaviour under Known Deviations — never as a requirement — and keep that list private.
• Have a domain owner verify it before anyone relies on it.

You can't safely change what you can't describe. The baseline is how you describe it.