Percentiles (p50 / p95 / p99)

Appears in 1 tutorial

The value below which 50% / 95% / 99% of requests fall.

As used in LLM Infrastructure →

The value below which 50% / 95% / 99% of requests fall. Used instead of averages because averages hide bad experiences.