Ask
344

cloudflare workers or a $7 vps for an api at 45 req/s and p95 under 300ms Benchmarks

Read-heavy JSON API. Steady 45 req/s with peaks around 120, so roughly 3.9M requests a day. Contract says p95 under 300ms measured at the client. Data lives in Postgres, single region, and about 70% of responses are cacheable for 60 seconds.

I can put this on Workers or on a $7/month VPS with 2 vCPU and 4GB. I am one person and I do not want to be woken up by a full disk.

What actually decides this - cost, latency, or the fact that nobody is on call?

9 answers Share
Report

Answering anonymously — a moderator will review it first.

  • @cast_iron_carl · 7mo ago · 3 replies

    Do the arithmetic before the architecture. 3.9M/day is about 117M requests a month.

    On the paid Workers plan that is roughly $5 base with 10M requests included, then 107M extra at $0.30 per million, so about $32, plus CPU time on top - at 5ms of CPU per request you are looking at another ten dollars or so. Call it $45-50/month. Check the current numbers yourself, but the shape does not move.

    The VPS is $7 and 45 req/s of cached JSON is nothing for 2 vCPU. So the money says VPS by a factor of six.

    The money is not the interesting part though. Your 70% cacheable is - stick a CDN in front of either option and 70% of your traffic never reaches your origin at all, which takes 45 req/s down to 14. At 14 req/s a $7 box is bored and the p95 question becomes entirely about the distance to Postgres, which is the same for both options because the database has not moved.

    One person, no on-call: I would take the VPS, put a CDN in front, and spend the difference on a managed database and backups I do not run myself.

    281
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @crimp_not_solder · 7mo ago

      The CDN point reframes it. I had been treating cacheability as a nice-to-have rather than as the thing that decides the capacity question.

      68
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
    • @cast_iron_carl · 7mo ago

      It usually is the thing. Most "do I need edge compute" arguments are actually cache-hit-rate arguments wearing a costume.

      77
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @two_vcpu_club · 7mo ago · 2 replies

    Disagreeing on the on-call part. The $7 VPS is cheap in dollars and not cheap in attention: unattended upgrades, a firewall, log rotation before the disk fills, TLS renewal you will find out has broken on a Sunday, and a deploy story you build yourself. None of it is hard, all of it is yours forever, and the failure mode is you being the only person who knows how it is wired.

    Workers deletes that entire column for about $40/month. Whether $40 is worth it depends on what an hour of your time is worth and how much you enjoy sysadmin work. For me it stopped being worth it the second time I debugged a full disk at midnight.

    176
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @cast_iron_carl · 7mo ago

      No argument with any of that, but note it applies to a VPS you administer by hand. A container on a managed host is $10-15 rather than $7, has no ssh, no disk to fill and no TLS to renew, and it is still a single region single process app that any of us could reason about. The gap between "box I patch" and "edge platform" has a lot of middle in it.

      71
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @adjunct_ora · 7mo ago

    Neither is a p95 risk at this volume - your p95 is going to be dominated by the Postgres round trip and whatever N+1 is hiding in the read path. Before choosing, put a Server-Timing header on the response splitting db time from handler time, run it for a day, and look at the p95 of each. I have watched a team migrate to the edge to save 40ms of compute while carrying a 220ms query.

    118
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @schema_drift_lu · 7mo ago

    The constraint people underrate: on Workers your database access has to work without a raw socket, which means an HTTP driver or a pooler in front. If your read path is a handful of queries that is fine. If there is a reporting endpoint doing something baroque, that endpoint is going to be the reason the migration takes three weeks instead of two days. Go and look at your ugliest query before deciding.

    74
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @maren_dowd · 7mo ago

    Whichever you pick, put the p95 in a check that pages you rather than in a contract nobody measures. "p95 under 300ms measured at the client" and "p95 under 300ms measured in my dashboard" are different numbers and the gap is where the argument with the customer happens in month four.

    52
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @pivot_pilot · 7mo ago

    $7 VPS, 3.9M requests a day. Somewhere a capacity planning consultant just felt a chill and does not know why.

    31
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report