node container exits 137 under load on a 2gb vps with mem_limit 512m Dockerfile
Small API, Node 22, running under compose on a 2 GB Hetzner box next to postgres and caddy. Under any real traffic the api container dies:
api Exited (137) 8 seconds ago
docker inspect --format '{{.State.OOMKilled}}' api says true. I set mem_limit: 512m in compose specifically to protect postgres.
RSS sits around 300 MB idle and climbs to about 540 under load, then it's killed. What I don't understand is why node doesn't garbage collect harder as it approaches the limit instead of walking straight into it. Is 512m just too small for Node, or am I holding this wrong?
@coworking_cass · 6h ago · 3 replies
Node doesn't know about your cgroup limit. V8 sizes its old-space heap from what it believes total system memory to be — on a 2 GB box that's roughly 1 GB — so it happily grows toward 1 GB while the cgroup kills you at 512 MB. It isn't failing to GC, it thinks it has plenty of room left.
Tell it the truth:
Leave headroom, because old space isn't the whole RSS — buffers, the code cache, native modules and stacks all live outside it. 384 under a 512 limit is about right; if you do a lot of Buffer work, leave more.
Then watch it. If it now GCs constantly and CPU spikes instead of dying, the limit really is too small and you raise it. Nine times out of ten this is the entire bug.
Reply
Report
@edge_runtime_bo · 13h ago
Two days. That's how long I spent reading my own code looking for a leak. Set it to 384, ran the same load test, peak RSS 470, no kills.
Reply
Report
@wonder_why_wren · 3h ago
Same class of bug hits the JVM, Go (GOMEMLIMIT) and Python with big allocators. Any runtime that autotunes off host memory needs telling when it's inside a cgroup.
Reply
Report