durable object alarms or a real queue for 5k scheduled reminders a day Queue Choice

A per-user Durable Object with a single alarm for its next due reminder, for exactly the reason you identified: the state that changes and the scheduling live in the same place, so an edit is just recomputing the next alarm time and calling setAlarm again.

Per user, not per reminder. Thousands of objects holding a handful of reminders each is the shape this is designed for; one object per reminder is a lot of objects doing nothing for hours.

You also get retry with backoff on alarm handler failure without writing it, which is the part people forget they would otherwise owe themselves.

Option B is not the embarrassing choice you think it is.

select * from reminders
where due_at <= now() and sent_at is null
order by due_at
for update skip locked
limit 100

once a minute. 5,000 a day is 3.5 a minute. One table, one query, one cron. Edits are an UPDATE. Cancels are a DELETE. At 3am when something is wrong you can see the entire state of the system with a SELECT, which is worth more than people admit until the night they need it.

Do not use delayed queue messages for anything editable. You end up keeping a cancellation table so the consumer can check "is this still wanted" on delivery - at which point you have built option B anyway, but with two sources of truth and a race between them.

Delayed messages are for things that are true when you enqueue them and stay true.

Whichever you pick, make the send idempotent on (reminder_id, scheduled_for). Every one of these designs fires twice eventually, and "user got two pushes" is a bug report you can avoid with a unique index.

The boring answer nobody wants: you are already running Postgres. Do B until it hurts. It will not hurt at 5k.