Inference benchmarking & load testing
Inference benchmarking & load testing covers tools that measure the speed, capacity and cost of inference engines and providers.
In brief
Inference benchmarking & load testing, in brief.
What it isInference benchmarking & load testing covers tools that measure the speed, capacity and cost of inference engines and providers.
Where it sitsOne of 13 categories in the Inference and serving layer. Inference and serving is everything that turns a trained model into a running service: hosted inference providers, gateways that route between them, serving engines, and the tools that make models smaller and faster.
What we track15 products, including AIPerf, fmperf, GenAI-Perf (Triton perf_analyzer), Grafana k6 and GuideLLM.
The products
15 products in Inference benchmarking & load testing.
Each name opens a search for that product in our index.
ProductMakerIts own site
From our census of 2026-11-08. A product listed here is one that belongs in this category; the search answer says whether we have checked it.
Questions
Inference benchmarking & load testing, answered.
What is Inference benchmarking & load testing in AI infrastructure?Inference benchmarking & load testing covers tools that measure the speed, capacity and cost of inference engines and providers.
Which layer of AI infrastructure is Inference benchmarking & load testing in?Inference benchmarking & load testing is in the Inference and serving layer. Inference and serving is everything that turns a trained model into a running service: hosted inference providers, gateways that route between them, serving engines, and the tools that make models smaller and faster.
Which products are in Inference benchmarking & load testing?We track 15 products in Inference benchmarking & load testing, including AIPerf, fmperf, GenAI-Perf (Triton perf_analyzer), Grafana k6, GuideLLM, inference-benchmarker, inference-perf and llm-load-test. The full list is on this page.
Inference and serving
The other categories in Inference and serving.
AI gateways & model routersBatch inferenceGenerative media inference platformsGenerative media workflow toolsInference kernels & attention librariesInference providersKV-cache & prompt-caching infrastructureLoRA serving & adapter hubsModel compilers & graph optimizersOn-device / edge runtimesQuantization / optimizationServing engines & model servers