On-device / edge runtimes
On-device / edge runtimes covers software runtimes that execute models on phones, laptops, browsers and edge chips (llama.cpp, ExecuTorch, MLC LLM, LiteRT, Ollama).
In brief
On-device / edge runtimes, in brief.
What it isOn-device / edge runtimes covers software runtimes that execute models on phones, laptops, browsers and edge chips (llama.cpp, ExecuTorch, MLC LLM, LiteRT, Ollama).
Where it sitsOne of 13 categories in the Inference and serving layer. Inference and serving is everything that turns a trained model into a running service: hosted inference providers, gateways that route between them, serving engines, and the tools that make models smaller and faster.
What we track36 products, including Apple Foundation Models framework, Cactus, Candle, Chrome built-in AI (Prompt API) and Core ML.
The products
36 products in On-device / edge runtimes.
Each name opens a search for that product in our index.
ProductMakerIts own site
From our census of 2026-11-08. A product listed here is one that belongs in this category; the search answer says whether we have checked it.
Questions
On-device / edge runtimes, answered.
What is On-device / edge runtimes in AI infrastructure?On-device / edge runtimes covers software runtimes that execute models on phones, laptops, browsers and edge chips (llama.cpp, ExecuTorch, MLC LLM, LiteRT, Ollama).
Which layer of AI infrastructure is On-device / edge runtimes in?On-device / edge runtimes is in the Inference and serving layer. Inference and serving is everything that turns a trained model into a running service: hosted inference providers, gateways that route between them, serving engines, and the tools that make models smaller and faster.
Which products are in On-device / edge runtimes?We track 36 products in On-device / edge runtimes, including Apple Foundation Models framework, Cactus, Candle, Chrome built-in AI (Prompt API), Core ML, ExecuTorch, Foundry Local / Windows ML and Gemini Nano via ML Kit GenAI / AICore. The full list is on this page.
Inference and serving
The other categories in Inference and serving.
AI gateways & model routersBatch inferenceGenerative media inference platformsGenerative media workflow toolsInference benchmarking & load testingInference kernels & attention librariesInference providersKV-cache & prompt-caching infrastructureLoRA serving & adapter hubsModel compilers & graph optimizersQuantization / optimizationServing engines & model servers