Twelve async workers fetching pages, all consulting a map of about 50k domain to rate-limit entries. Reads are maybe 95% of accesses, writes happen when a new domain appears or a limit updates. I can either wrap it in Arc<RwLock<HashMap>> and share it, or give ownership to one task and talk to it over channels. Memory is not a constraint, latency per request is. Which way would you go and why?
Third option worth pricing: a sharded concurrent map crate. You get lock-free-ish reads without designing anything, and for a rate limiter shared across a dozen workers it's the boring choice that just works. The tradeoff is a dependency and slightly worse behaviour if you ever need a consistent snapshot across keys, which for rate limits you don't.
This is what I'd do. Went from RwLock to a sharded map on a similar workload and the p99 got noticeably tighter, mostly because writes stopped occasionally blocking a burst of readers.
That p99 point is the sort of thing I wouldn't have caught in a benchmark of averages. Thanks.
Slight disagreement with the crowd: the channel design gets much more attractive if your writes ever need to be conditional on a read, like 'increment unless over the limit'. With a lock you either take a write lock for every check or you race. Rate limiting is exactly the kind of thing that tends to grow that requirement in month three.
Fair, though you can also express that as a single write-locked method on a wrapper type rather than a message. Same atomicity, no round trip.
Whichever you pick, hide it behind a small struct with two or three methods rather than passing the map around. Then swapping the implementation later is an afternoon rather than a refactor, and you can benchmark both properly.
50k entries is small enough that you could also just clone the map into each worker and rebuild it periodically, if staleness of a few seconds is acceptable for a rate limit. Not elegant, zero contention.
@borrowck_ben · 4w ago · 3 replies
With 95% reads and twelve workers I'd start with
Arc<RwLock<HashMap>>using the async RwLock only if you hold it across awaits, and the std one if you don't. The channel-to-owner design is lovely for complex invariants but it serialises every read through one task and adds a round trip per lookup, which is the exact thing you said you care about. Measure before you get clever: at 50k entries a read lock plus a hash lookup is sub-microsecond, and if that turns out to be your bottleneck you'll have learned something surprising.Reply
Report
@borrowck_ben · last mo.
Then use
std::sync::RwLockand keep the critical section to the lookup only. Async mutexes are slower and you don't need what they offer.Reply
Report
@yaml_yusuf · 4w ago
Good point on holding across awaits, I don't need to, the lookup returns a small Copy struct.
Reply
Report