The migration that almost killed our quarter.
A retrospective on a six-week database move that became a four-month one, and what I learned about scope, splash damage, and the very specific kind of optimism that makes engineers dangerous.
The migration was supposed to take six weeks. It took seventeen. By week ten, the question wasn’t whether we’d finish — it was whether the people doing the work would still be at the company when it shipped.
How we got the estimate wrong
We estimated based on the happy path — a clean cutover of a single primary, four downstream consumers, and one nightly job. What we didn’t see was the long tail of code paths that read from the database in unobvious ways: a cron job nobody owned anymore, a debug endpoint that loaded a half-million rows, a third-party integration that pinned itself to an old schema version through sheer inertia.
Each of these added a day. Most of those days came at the worst possible moments.
The optimism that hurt us
The most expensive sentence of the project was this: “we’ll figure that out when we get there.” Said five times in planning, it added five surprises to a project that needed exactly zero. Each surprise was a person, working on their own at 9 p.m., learning something they wished they’d known a month earlier.
If I’d written down every “we’ll figure that out” sentence at the start and forced an answer for each one, the project would have looked harder. We probably would have descoped. The quarter would have lived.
What I’ll do next time
A small ritual I now run before every migration: list every code path that touches the thing being migrated, including the ones nobody currently owns, and assign a person to each. The result is always uncomfortable. That’s the point.