Latency
Latency is the time from sending a request to receiving the response, measured with percentiles (p50/p95/p99) to capture tail latency.
Purpose
Quantify the user-experience-critical metric and optimize tail latency.
Mechanism
p50: the median. p95/p99: tail latency (affects few users but noticeably). Averages get skewed by the long tail. Tail latency stems from slow requests, GC, lock contention, network jitter. p99 < 100ms is a common SLO.
Use cases
- Performance optimization
- SLO
- User experience