KV-cache & prompt-caching infrastructure
KV-cache & prompt-caching infrastructure covers systems that store and reuse a model's earlier work across requests, so repeated prompts cost less and answer faster.
In brief
KV-cache & prompt-caching infrastructure, in brief.
What it isKV-cache & prompt-caching infrastructure covers systems that store and reuse a model's earlier work across requests, so repeated prompts cost less and answer faster.
Where it sitsOne of 13 categories in the Inference and serving layer. Inference and serving is everything that turns a trained model into a running service: hosted inference providers, gateways that route between them, serving engines, and the tools that make models smaller and faster.
What we track24 products, including AIBrix (distributed KV cache), Amazon Bedrock prompt caching, Anthropic prompt caching, Gemini API context caching and GPTCache.
The products
24 products in KV-cache & prompt-caching infrastructure.
Each name opens a search for that product in our index.
ProductMakerIts own site
From our census of 2026-11-08. A product listed here is one that belongs in this category; the search answer says whether we have checked it.
Questions
KV-cache & prompt-caching infrastructure, answered.
What is KV-cache & prompt-caching infrastructure in AI infrastructure?KV-cache & prompt-caching infrastructure covers systems that store and reuse a model's earlier work across requests, so repeated prompts cost less and answer faster.
Which layer of AI infrastructure is KV-cache & prompt-caching infrastructure in?KV-cache & prompt-caching infrastructure is in the Inference and serving layer. Inference and serving is everything that turns a trained model into a running service: hosted inference providers, gateways that route between them, serving engines, and the tools that make models smaller and faster.
Which products are in KV-cache & prompt-caching infrastructure?We track 24 products in KV-cache & prompt-caching infrastructure, including AIBrix (distributed KV cache), Amazon Bedrock prompt caching, Anthropic prompt caching, Gemini API context caching, GPTCache, InfiniStore, kvcached and llm-d KV cache manager. The full list is on this page.
Inference and serving
The other categories in Inference and serving.
AI gateways & model routersBatch inferenceGenerative media inference platformsGenerative media workflow toolsInference benchmarking & load testingInference kernels & attention librariesInference providersLoRA serving & adapter hubsModel compilers & graph optimizersOn-device / edge runtimesQuantization / optimizationServing engines & model servers