Ask
87
@migration_marek ·

drizzle-kit generate turned a column rename into drop + add and i ran it on staging Migrations

Renamed name to full_name in the schema file, generated, applied to staging in the deploy script. The SQL was a DROP COLUMN followed by an ADD COLUMN. 41,802 rows of names are now NULL and the migration exited 0 with no warning. Staging so nobody died, but the same script runs against prod on Thursday. How do people stop this happening?

7 answers Share
Report

Answering anonymously — a moderator will review it first.

  • @nadia_brill · last wk. · 2 replies

    The generator diffs two schema snapshots. A rename and a drop-plus-add produce an identical diff, so it has to ask you — and drizzle-kit does ask, interactively. If you run it non-interactively in a script, or you hit enter through the prompt, you get the destructive branch by default.

    Three rules that have kept me out of this:

    • Generate on your machine, interactively, never in CI.
    • Open the .sql before committing it. Every time. It is thirty seconds.
    • Make the pipeline grep the pending migrations for DROP COLUMN and DROP TABLE and fail unless the PR has an explicit label. Destructive changes should require a human sentence somewhere.
    96
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @migration_marek · last wk.

      It was being generated inside the deploy script with no tty, which I inherited and never questioned. Moved generation to local, added the grep. Thursday looks a lot less exciting now.

      20
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @pentalobe_junie · last wk. · 2 replies

    For production, do not rename columns at all. Expand and contract:

    1. Add full_name, nullable.
    2. Backfill in batches.
    3. Deploy code that writes both and reads full_name.
    4. A release later, drop name.

    There is nothing for the generator to guess about, no prompt to misread, and every step is independently revertible. It is four migrations instead of one and it is worth it every single time.

    61
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @gradschool_gary · last wk.

      The underrated part is rollback. With expand/contract you can redeploy the previous version of the app at any point and it still works, because the old column is still there. A rename makes your migration and your deploy a single atomic thing you cannot undo separately.

      24
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @wild_ferment · 7d ago

    Recovery, if you still want the data: most managed Postgres has point-in-time restore or branching. Restore to a branch at a timestamp before the migration, then UPDATE ... FROM joining on id to copy the column back. Ten minutes of work if the restore is available, and it is worth doing on staging just to rehearse the procedure.

    22
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @dockerfile_yuki · last wk.

    Related rule: push points at your own dev database and nothing else, ever. It is a dev-loop convenience that skips the review step entirely, which is precisely the step that would have caught this.

    37
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @leech_wrangler · last wk.

    Good news: you found it on staging. Bad news: staging found it for you because staging has data worth losing, which most people's does not, which is why most people find it in prod.

    8
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report