i can see p95 creeping up but i cannot tell whether it is a handful of cold invocations dragging the tail or my handler genuinely getting slower. the platform dashboard gives me duration and invocation count and nothing about which requests were cold. before i start optimising i want to know how many of my requests are actually paying init cost. how do you measure the split in practice?
module scope boolean, flipped at the end of the first handler run. log it on every request as cold true or false along with the age of the instance. ten lines, works on every platform, and it survives you changing providers.
one caution: platform duration is not what the user experiences. it excludes queue time and the network, and on some platforms it starts counting after the runtime is ready, which is the exact thing you are trying to measure. put a server timing header on the response and collect the real number from the client, then compare it to the platform metric. mine differed by a couple of hundred milliseconds and always in the same direction.
agreed, and that gap is the honest argument for measuring both. platform metrics tell you what to fix, client timing tells you whether it mattered.
when i did this on a service doing a few hundred thousand requests a day, roughly six percent of invocations were cold and they did not move the median at all. p50 was flat, p99 was almost double the warm p99, and the entire alert that started the investigation was one tail metric. knowing the cold rate told me whether it was worth engineering away, which for six percent it was not.
i spent a week shaving a handler that turned out to be a fifth of the request. init was doing the work, i was optimising the part i could see, and the only reason i noticed was that someone asked me to prove the improvement.
count cold invocations, not milliseconds. if the rate is low the tail is not your problem, it is a graph shape.
@shadowing_sena · 2mo ago · 3 replies
on lambda the report line for an invocation includes an init duration field only when that invocation was cold, so counting the report lines that carry it gives you the cold rate directly and the init cost separately from the billed duration. if you are somewhere without that, the portable trick is to record a timestamp at module scope and compare it to the time at the top of the handler on the first request the instance serves. either way, tag the log line so you can group by cold and warm and stop reasoning about one average.
Reply
Report
@salted_hash_h · 2mo ago
and once you split them the tail usually stops being mysterious. mine was two clean distributions that looked like one ugly one.
Reply
Report
@first_repo_finn · 2mo ago
grouping by cold and warm rather than looking at one p95 is the reframe i needed. i have been averaging two different populations together.
Reply
Report