Why the average lies
Consider a hundred requests where ninety-five complete in 120ms and five take 3 seconds. The mean is around 264ms, which looks acceptable on a dashboard. But one in twenty of your users just waited three seconds, and they are the ones who will complain, abandon a cart, or churn.
Percentiles preserve that information. p95 answers a question the average cannot: how bad is it for the unlucky ones?
Which percentile to use
- p50 (median)The typical experience. Useful for spotting broad regressions that affect everyone.
- p95The usual choice for SLOs. Catches real pain without letting a handful of extreme outliers dominate your alerting.
- p99The tail. Important at scale — at a million requests a day, p99 is ten thousand slow experiences.
- maxAlmost always noise. One timeout, one garbage-collection pause, and your max is meaningless as a trend.
You cannot take the p95 of five servers and average them to get a fleet p95. That is a genuinely different number. Percentiles must be computed from the combined raw distribution, which is why aggregation strategy matters in monitoring tooling.
Setting a latency budget
Once you know your current p95, the practical next step is turning it into a threshold your monitoring enforces. Pick a number slightly above today's p95 — enough headroom that normal variance does not alarm, tight enough that a genuine regression trips it.
Then treat a breach as a failure rather than a note. A response that arrives after your client has given up did not succeed, whatever status code it carried.