Expand, then contract: shipping schema changes without a maintenance window

There is a window in every rolling deploy, usually somewhere between forty seconds and four minutes, where two releases of your application are both serving traffic against one database. The load balancer is draining the old containers. The new ones are taking connections. Both are running queries.

Most of the time nobody thinks about that window, because most changes do not care about it. Schema changes care about it enormously, and the difference between a change that is invisible and a change that takes the site down for four minutes is not the size of the change. It is the direction.

Additive is invisible. Destructive is not.

Think about what the old container is actually sending during the overlap.

An ORM does not send SELECT *. It sends an explicit column list, generated from the model definitions that were compiled into that container. So:

Adding a nullable column is invisible. The old container’s queries never name it. The new container’s queries do, and the column is there. Nobody notices. The same is true of adding a table, or adding an index.

Dropping or renaming a column is fatal. The old container is still sending queries that name it. Every one of them errors — not the ones touching the new feature, every query against that table, including the login path. You have not degraded a feature. You have taken out a model for the length of the drain.

There is a third category people forget. Tightening a constraint is fatal in the other direction. Make a column NOT NULL and the old release, which does not know to populate it, starts failing on insert. Same for a new unique constraint the old code can violate, and same for a foreign key it does not maintain.

And there is a fourth, which is not about the overlap at all but ends up in the same incident channel: the migration itself takes a lock. A statement that rewrites a table holds an exclusive lock, and every query behind it queues. Worse, the lock request itself queues behind existing readers and then blocks everything that arrives after it. A migration that runs in 200ms on your laptop can hold production for a minute because it sat behind one long-running report.

The three-step pattern

The fix is old and well documented and almost nobody does it consistently, which is the interesting part. It is three releases, not one.

1. Expand. Add the new thing, nullable, with no constraint. Add the new table. Create the index concurrently so it does not lock writes. Ship it. Old code is untouched because it never refers to any of it. This release is safe to roll back trivially, because rolling back code never requires rolling back an additive schema change.

2. Migrate and dual-write. Backfill the existing rows in batches — not one statement, because one statement is one long transaction and one long lock. Meanwhile the application writes both the old field and the new one, and reads the new one with a fallback to the old. This release is where the actual behaviour changes, and it is entirely reversible: the old column is still current, so rolling back loses nothing.

3. Contract. In a later release, once every container in the fleet is past step two and the backfill is verified complete, drop the old column and remove the dual-write. Now there is no old container that names it, because there is no old container.

Every one of those steps is deployable with no window. The pattern is not clever. It just requires you to have three deploys where you wanted one.

Documenting it does not work

We wrote this down. It was in the engineering guide, it was in the pull request template, and it worked for about five months. Then someone renamed a field in a model, ran makemigrations, got a clean RenameField operation, and shipped it on a Thursday. The review passed because the diff looked correct — it was correct, in the sense that the schema afterwards was the schema we wanted.

The problem with a documented convention is the same problem as any opt-in safety property: the unsafe version looks completely normal in review, and it only fails in an environment nobody is looking at when the pull request is open.

So we moved it into the deploy. A management command runs before migrate, inspects every unapplied migration, and fails non-zero if it finds an operation that the old release cannot survive:

DESTRUCTIVE = (RemoveField, DeleteModel, RenameField, RenameModel, AlterUniqueTogether)

for migration in unapplied:
    if reason := getattr(migration, "expand_contract_exempt", None):
        continue                     # declared, on purpose, with a why
    for op in migration.operations:
        if isinstance(op, DESTRUCTIVE) or tightens_a_constraint(op):
            fail(f"{migration.name}: {type(op).__name__} needs a contract release")

A genuine contract migration — step three, the one that is supposed to drop the column — declares itself:

expand_contract_exempt = "PayrollComponent.legacy_rate; dual-write shipped in 0142, backfill verified 2026-08-19"

That string is doing three jobs. It makes the exemption deliberate rather than accidental. It puts the justification next to the code instead of in a chat thread. And because it is a string with a fixed name, grep answers “show me every destructive migration we have ever declared” in one command.

If a change genuinely needs downtime — and a few do — the deploy runs with an explicit destructive flag, which is a decision someone makes with their hands, at a time they chose.

What the check cannot see

Two honest limitations, because a gate you trust more than it deserves is worse than no gate.

It reads migration operations, not SQL. A RunSQL or a RunPython can do anything at all, and the checker has no idea what is inside it. We treat raw operations as suspect and require the same declaration, which is a blunt instrument that occasionally annoys people writing a harmless data fix.

It also cannot see application-level compatibility. Changing the meaning of an existing column — repurposing a status value, changing a unit from days to hours — is additive as far as the schema is concerned and completely breaks the old release. No schema checker catches semantic changes. That one still needs a human who knows what the column is for.

The step everyone forgets

Here is the part that does not appear in the tidy version of this advice.

Nobody comes back for step three. The expand ships because the feature needs it. The dual-write ships because the feature needs it. The contract ships because… nothing. The feature works. The old column sits there, populated, ignored, alongside a dual-write nobody will delete and a fallback read nobody will simplify. Two years later a new engineer asks which of these two columns is the real one and gets three different answers.

We have not solved this. What we do is smaller than a solution: the contract migration is written at the same time as the expand, committed on a branch that does not merge, and referenced by number in the dual-write code. When someone opens that file, the follow-up is already written and the decision is only whether it is safe to merge yet.

It still slips. The list is shorter than it used to be, and every entry on it has a name and a reason attached, which is more than we could say before. If somebody has a better answer than a list somebody has to read, we would genuinely like to hear it.